Jul 02 19:11:26.152165 ip-10-0-132-104 systemd[1]: Starting Kubernetes Kubelet... Jul 02 19:11:26.578677 ip-10-0-132-104 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 19:11:26.578677 ip-10-0-132-104 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 19:11:26.578677 ip-10-0-132-104 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 19:11:26.578677 ip-10-0-132-104 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 19:11:26.578677 ip-10-0-132-104 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 19:11:26.578677 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.542342 2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 19:11:26.578677 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546731 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:11:26.578677 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546746 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:11:26.578677 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546750 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:11:26.578677 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546753 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:11:26.578677 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546756 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:11:26.578677 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546759 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:11:26.578677 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546762 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:11:26.578677 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546765 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546767 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546770 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546776 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546779 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546782 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546784 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546787 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546789 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546792 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546794 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546797 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546800 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546803 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546806 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546808 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546811 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546813 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546816 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546819 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:11:26.580780 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546822 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:11:26.583123 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546824 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:11:26.583123 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546827 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:11:26.583123 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546830 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:11:26.583123 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546833 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:11:26.583123 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546836 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:11:26.583123 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546839 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:11:26.583123 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546841 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:11:26.583123 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546844 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:11:26.583123 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546847 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:11:26.583123 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546849 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:11:26.583123 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546852 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:11:26.583123 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546855 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:11:26.583123 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546857 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:11:26.583123 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546862 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:11:26.583123 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546866 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:11:26.583123 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546869 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:11:26.583123 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546889 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:11:26.583123 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546893 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:11:26.583123 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546897 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546902 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546905 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546908 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546911 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546914 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546918 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546920 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546923 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546926 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546929 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546931 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546934 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546938 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546941 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546943 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546946 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546949 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546951 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546954 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:11:26.597001 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546956 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:11:26.599210 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546959 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:11:26.599210 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546962 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:11:26.599210 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546966 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:11:26.599210 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546968 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:11:26.599210 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546974 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:11:26.599210 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546978 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:11:26.599210 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546980 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:11:26.599210 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546983 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:11:26.599210 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546985 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:11:26.599210 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546988 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:11:26.599210 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546990 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:11:26.599210 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546993 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:11:26.599210 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546996 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:11:26.599210 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.546998 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:11:26.599210 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547001 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:11:26.599210 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547004 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:11:26.599210 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547006 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:11:26.599210 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547009 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:11:26.599210 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547011 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:11:26.600984 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547014 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:11:26.600984 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547016 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:11:26.600984 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547019 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:11:26.600984 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547022 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:11:26.600984 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547026 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:11:26.600984 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547030 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:11:26.600984 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547034 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:11:26.600984 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547037 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:11:26.600984 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547039 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:11:26.600984 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547042 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:11:26.600984 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547045 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:11:26.600984 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547048 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:11:26.600984 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547051 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:11:26.600984 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547760 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:11:26.600984 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547766 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:11:26.600984 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547769 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:11:26.600984 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547772 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:11:26.600984 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547776 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:11:26.600984 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547780 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:11:26.603277 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547784 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:11:26.603277 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547789 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:11:26.603277 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547792 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:11:26.603277 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547794 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:11:26.603277 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547801 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:11:26.603277 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547804 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:11:26.603277 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547807 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:11:26.603277 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547810 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:11:26.603277 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547813 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:11:26.603277 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547816 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:11:26.603277 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547818 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:11:26.603277 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547821 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:11:26.603277 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547824 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:11:26.603277 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547826 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:11:26.603277 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547829 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:11:26.603277 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547831 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:11:26.603277 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547834 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:11:26.603277 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547836 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:11:26.603277 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547839 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547842 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547845 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547857 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547860 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547863 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547865 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547868 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547890 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547894 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547896 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547899 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547901 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547904 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547907 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547910 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547913 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547916 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547920 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547923 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:11:26.616972 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547926 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547928 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547931 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547933 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547936 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547939 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547941 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547944 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547946 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547949 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547951 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547954 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547957 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547959 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547962 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547964 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547967 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547970 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547972 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547976 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:11:26.619156 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547980 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:11:26.621163 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547983 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:11:26.621163 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547986 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:11:26.621163 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547989 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:11:26.621163 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547992 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:11:26.621163 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547994 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:11:26.621163 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.547997 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:11:26.621163 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548000 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:11:26.621163 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548003 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:11:26.621163 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548007 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:11:26.621163 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548010 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:11:26.621163 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548014 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:11:26.621163 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548017 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:11:26.621163 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548019 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:11:26.621163 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548022 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:11:26.621163 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548026 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:11:26.621163 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548029 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:11:26.621163 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548032 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:11:26.621163 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548035 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:11:26.621163 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548038 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:11:26.620590 ip-10-0-132-104 systemd[1]: Started Kubernetes Kubelet. Jul 02 19:11:26.622742 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548040 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:11:26.622742 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548043 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:11:26.622742 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548046 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:11:26.622742 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548048 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:11:26.622742 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548051 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:11:26.622742 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548054 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:11:26.622742 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548056 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:11:26.622742 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548059 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:11:26.622742 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548061 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:11:26.622742 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548064 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:11:26.622742 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548066 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:11:26.622742 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548068 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:11:26.622742 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548071 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:11:26.622742 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548074 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:11:26.622742 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548077 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:11:26.622742 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548478 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:11:26.622742 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548482 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:11:26.622742 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548485 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:11:26.622742 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548488 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548491 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548494 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548496 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548500 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548504 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548507 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548510 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548513 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548515 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548518 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548520 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548523 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548525 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548528 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548530 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548533 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548536 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548538 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548541 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:11:26.638755 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548544 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:11:26.655485 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548546 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:11:26.655485 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548549 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:11:26.655485 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548551 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:11:26.655485 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548554 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:11:26.655485 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548556 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:11:26.655485 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548559 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:11:26.655485 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548561 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:11:26.655485 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548564 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:11:26.655485 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548566 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:11:26.655485 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548569 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:11:26.655485 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548571 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:11:26.655485 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548574 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:11:26.655485 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548576 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:11:26.655485 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548579 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:11:26.655485 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548581 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:11:26.655485 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548586 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:11:26.655485 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548590 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:11:26.655485 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548592 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:11:26.655485 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548595 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:11:26.668980 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548597 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:11:26.668980 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548600 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:11:26.668980 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548603 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:11:26.668980 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548605 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:11:26.668980 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548608 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:11:26.668980 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548610 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:11:26.668980 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548613 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:11:26.668980 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548617 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:11:26.668980 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548620 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:11:26.668980 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548623 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:11:26.668980 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548626 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:11:26.668980 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548630 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:11:26.668980 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548633 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:11:26.668980 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548636 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:11:26.668980 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548639 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:11:26.668980 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548642 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:11:26.668980 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548644 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:11:26.668980 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548647 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:11:26.668980 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548649 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548652 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548654 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548657 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548659 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548662 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548664 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548667 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548670 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548673 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548676 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548680 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548683 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548685 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548688 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548690 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548693 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548695 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548697 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548700 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:11:26.671886 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548703 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548705 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548708 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548710 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548712 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548715 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548718 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548720 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548723 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548726 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548729 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548732 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548735 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548737 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548740 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548742 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548744 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.548747 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548825 2556 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548834 2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548840 2556 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 19:11:26.684989 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548844 2556 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548850 2556 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548855 2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548859 2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548863 2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548866 2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548869 2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548890 2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548894 2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548897 2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548900 2556 flags.go:64] FLAG: --cgroup-root="" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548903 2556 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548906 2556 flags.go:64] FLAG: --client-ca-file="" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548909 2556 flags.go:64] FLAG: --cloud-provider="external" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548912 2556 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548917 2556 flags.go:64] FLAG: --cluster-domain="" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548920 2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548923 2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548926 2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548929 2556 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548933 2556 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548936 2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548939 2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 19:11:26.690656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548942 2556 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548945 2556 flags.go:64] FLAG: --contention-profiling="false" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548948 2556 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548951 2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548955 2556 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548958 2556 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548962 2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548965 2556 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548968 2556 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548970 2556 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548975 2556 flags.go:64] FLAG: --enable-server="true" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548980 2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548985 2556 flags.go:64] FLAG: --event-burst="100" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548988 2556 flags.go:64] FLAG: --event-qps="50" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548991 2556 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548994 2556 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.548997 2556 flags.go:64] FLAG: --eviction-hard="" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549001 2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549004 2556 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549007 2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549009 2556 flags.go:64] FLAG: --eviction-soft="" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549012 2556 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549015 2556 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549018 2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549021 2556 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 19:11:26.709779 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549024 2556 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549027 2556 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549030 2556 flags.go:64] FLAG: --feature-gates="" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549033 2556 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549036 2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549039 2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549042 2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549046 2556 flags.go:64] FLAG: --healthz-port="10248" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549048 2556 flags.go:64] FLAG: --help="false" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549051 2556 flags.go:64] FLAG: --hostname-override="ip-10-0-132-104.ec2.internal" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549054 2556 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549057 2556 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549060 2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549063 2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549066 2556 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549069 2556 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549072 2556 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549076 2556 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549081 2556 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549085 2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549088 2556 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549092 2556 flags.go:64] FLAG: --kube-reserved="" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549095 2556 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549098 2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 19:11:26.857870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549101 2556 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549103 2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549106 2556 flags.go:64] FLAG: --lock-file="" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549109 2556 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549112 2556 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549115 2556 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549120 2556 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549123 2556 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549126 2556 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549129 2556 flags.go:64] FLAG: --logging-format="text" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549131 2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549135 2556 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549137 2556 flags.go:64] FLAG: --manifest-url="" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549140 2556 flags.go:64] FLAG: --manifest-url-header="" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549145 2556 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549148 2556 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549152 2556 flags.go:64] FLAG: --max-pods="110" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549156 2556 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549159 2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549162 2556 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549164 2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549167 2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549170 2556 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549173 2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549181 2556 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 19:11:26.861663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549185 2556 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549189 2556 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549192 2556 flags.go:64] FLAG: --pod-cidr="" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549196 2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549201 2556 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549204 2556 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549207 2556 flags.go:64] FLAG: --pods-per-core="0" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549210 2556 flags.go:64] FLAG: --port="10250" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549213 2556 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549217 2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0af3fa71eab78a8c5" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549220 2556 flags.go:64] FLAG: --qos-reserved="" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549222 2556 flags.go:64] FLAG: --read-only-port="10255" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549225 2556 flags.go:64] FLAG: --register-node="true" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549228 2556 flags.go:64] FLAG: --register-with-taints="" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549232 2556 flags.go:64] FLAG: --registry-burst="10" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549235 2556 flags.go:64] FLAG: --registry-qps="5" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549237 2556 flags.go:64] FLAG: --reserved-cpus="" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549240 2556 flags.go:64] FLAG: --reserved-memory="" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549244 2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549247 2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549250 2556 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549253 2556 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549255 2556 flags.go:64] FLAG: --runonce="false" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549258 2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549261 2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 19:11:26.862684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549264 2556 flags.go:64] FLAG: --seccomp-default="false" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549266 2556 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549269 2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549272 2556 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549275 2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549278 2556 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549281 2556 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549284 2556 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549292 2556 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549295 2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549298 2556 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549301 2556 flags.go:64] FLAG: --system-cgroups="" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549304 2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549310 2556 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549313 2556 flags.go:64] FLAG: --tls-cert-file="" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549316 2556 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549320 2556 flags.go:64] FLAG: --tls-min-version="" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549322 2556 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549325 2556 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549328 2556 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549331 2556 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549333 2556 flags.go:64] FLAG: --v="2" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549337 2556 flags.go:64] FLAG: --version="false" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549341 2556 flags.go:64] FLAG: --vmodule="" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549345 2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 19:11:26.864618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.549348 2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549468 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549472 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549475 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549477 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549480 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549483 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549486 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549488 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549491 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549494 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549497 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549500 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549502 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549506 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549511 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549514 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549517 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549520 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549523 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:11:26.865397 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549526 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549528 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549531 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549533 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549536 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549538 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549541 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549544 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549546 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549549 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549551 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549554 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549556 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549558 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549561 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549563 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549566 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549569 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549571 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549574 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:11:26.865964 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549576 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549578 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549581 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549583 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549586 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549588 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549590 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549597 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549599 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549602 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549604 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549607 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549610 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549612 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549615 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549617 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549620 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549622 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549625 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549628 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:11:26.866550 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549630 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:11:26.867120 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549632 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:11:26.867120 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549635 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:11:26.867120 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549638 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:11:26.867120 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549642 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:11:26.867120 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549645 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:11:26.867120 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549648 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:11:26.867120 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549651 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:11:26.867120 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549654 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:11:26.867120 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549656 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:11:26.867120 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549659 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:11:26.867120 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549661 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:11:26.867120 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549664 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:11:26.867120 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549666 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:11:26.867120 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549669 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:11:26.867120 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549671 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:11:26.867120 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549674 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:11:26.867120 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549677 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:11:26.867120 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549681 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:11:26.867120 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549687 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549690 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549693 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549695 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549697 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549700 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549703 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549705 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549708 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549710 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549713 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549715 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549717 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549720 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549724 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549728 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549731 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549734 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549737 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549739 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:11:26.867675 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.549742 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:11:26.868251 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.550346 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 19:11:26.868251 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.557499 2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 19:11:26.868251 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.557517 2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 19:11:26.868251 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.557536 2556 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 19:11:26.868251 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.557542 2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 19:11:26.868251 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557616 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:11:26.868251 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557620 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:11:26.868251 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557623 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:11:26.868251 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557627 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:11:26.868251 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557629 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:11:26.868251 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557632 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:11:26.868251 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557635 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:11:26.868251 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557637 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:11:26.868251 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557640 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:11:26.868251 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557643 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:11:26.956855 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557645 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:11:26.956855 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557648 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:11:26.956855 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557657 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:11:26.956855 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557659 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:11:26.956855 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557662 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:11:26.956855 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557665 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:11:26.956855 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557668 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:11:26.956855 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557670 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:11:26.956855 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557673 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:11:26.956855 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557676 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:11:26.956855 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557681 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:11:26.956855 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557685 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:11:26.956855 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557687 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:11:26.956855 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557690 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:11:26.956855 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557693 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:11:26.956855 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557695 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:11:26.956855 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557698 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:11:26.956855 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557700 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:11:26.956855 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557703 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557705 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557708 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557710 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557716 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557719 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557722 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557724 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557727 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557729 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557732 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557734 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557737 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557739 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557742 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557744 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557749 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557758 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557761 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557764 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:11:26.957535 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557766 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557769 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557772 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557774 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557778 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557782 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557785 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557787 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557790 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557793 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557795 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557797 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557800 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557802 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557805 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557808 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557810 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557812 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557815 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557817 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:11:26.958292 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557820 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:11:26.958836 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557822 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:11:26.958836 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557825 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:11:26.958836 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557827 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:11:26.958836 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557830 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:11:26.958836 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557832 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:11:26.958836 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557836 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:11:26.958836 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557840 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:11:26.958836 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557843 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:11:26.958836 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557845 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:11:26.958836 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557854 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:11:26.958836 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557857 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:11:26.958836 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557859 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:11:26.958836 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557862 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:11:26.958836 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557864 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:11:26.958836 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557867 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:11:26.958836 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557889 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:11:26.958836 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557894 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:11:26.958836 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557896 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:11:26.958836 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557899 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:11:26.959433 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557903 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:11:26.959433 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557906 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:11:26.959433 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557909 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:11:26.959433 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557912 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:11:26.959433 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557915 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:11:26.959433 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557918 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:11:26.959433 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557920 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:11:26.959433 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557923 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:11:26.959433 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557925 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:11:26.959433 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557928 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:11:26.959433 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.557930 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:11:26.959433 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.557936 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 19:11:26.959433 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558093 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:11:26.959433 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558098 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558101 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558104 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558106 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558109 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558112 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558114 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558117 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558119 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558122 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558130 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558133 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558136 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558138 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558141 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558145 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558147 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558151 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558153 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558156 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:11:26.959868 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558158 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:11:26.960498 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558161 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:11:26.960498 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558164 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:11:26.960498 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558166 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:11:26.960498 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558169 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:11:26.960498 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558172 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:11:26.960498 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558175 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:11:26.960498 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558177 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:11:26.960498 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558180 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:11:26.960498 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558183 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:11:26.960498 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558185 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:11:26.960498 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558188 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:11:26.960498 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558190 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:11:26.960498 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558193 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:11:26.960498 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558195 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:11:26.960498 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558197 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:11:26.960498 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558200 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:11:26.960498 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558203 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:11:26.960498 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558205 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:11:26.960498 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558208 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:11:26.961030 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558210 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:11:26.961030 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558213 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:11:26.961030 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558215 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:11:26.961030 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558223 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:11:26.961030 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558226 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:11:26.961030 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558228 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:11:26.961030 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558232 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:11:26.961030 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558237 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:11:26.961030 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558239 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:11:26.961030 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558244 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:11:26.961030 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558246 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:11:26.961030 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558249 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:11:26.961030 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558251 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:11:26.961030 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558255 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:11:26.961030 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558258 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:11:26.961030 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558261 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:11:26.961030 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558264 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:11:26.961030 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558267 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:11:26.961030 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558270 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558272 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558275 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558278 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558281 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558283 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558286 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558288 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558290 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558293 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558296 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558298 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558300 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558303 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558305 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558308 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558310 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558313 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558325 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558328 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:11:26.961593 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558330 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:11:26.962199 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558334 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:11:26.962199 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558337 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:11:26.962199 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558341 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:11:26.962199 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558343 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:11:26.962199 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558346 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:11:26.962199 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558348 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:11:26.962199 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558351 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:11:26.962199 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558353 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:11:26.962199 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558356 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:11:26.962199 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558358 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:11:26.962199 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558361 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:11:26.962199 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558363 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:11:26.962199 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558366 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:11:26.962199 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558368 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:11:26.962199 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558371 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:11:26.962199 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558373 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:11:26.962199 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558376 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:11:26.962199 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558378 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:11:26.962199 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:26.558381 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:11:26.962758 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.558386 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 19:11:26.962758 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.558523 2556 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 19:11:26.962758 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.560686 2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 19:11:26.962758 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.561887 2556 server.go:1019] "Starting client certificate rotation" Jul 02 19:11:26.962758 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.562739 2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 19:11:26.962758 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.563137 2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 19:11:26.962758 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.564540 2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 19:11:26.962758 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.568569 2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 19:11:26.962758 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.580256 2556 log.go:25] "Validated CRI v1 runtime API" Jul 02 19:11:26.962758 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.584328 2556 log.go:25] "Validated CRI v1 image API" Jul 02 19:11:26.962758 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.586091 2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 19:11:26.962758 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.590415 2556 fs.go:135] Filesystem UUIDs: map[19d76c7f-844a-4307-8929-41b4b246a001:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a7692bbd-257f-4ff3-9f6e-9eb547feeab7:/dev/nvme0n1p4] Jul 02 19:11:26.963122 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.590432 2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 19:11:26.963230 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.595588 2556 manager.go:216] Machine: {Timestamp:2026-07-02 19:11:26.594376842 +0000 UTC m=+0.326373511 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099731 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2464f77c62fc9580e76795645ffe4a SystemUUID:ec2464f7-7c62-fc95-80e7-6795645ffe4a BootID:4c6612a5-a0a4-4463-86ab-bf0d01430bf3 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:84:9b:ef:e0:b9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:84:9b:ef:e0:b9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:a2:c7:5f:d0:45 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 19:11:26.963230 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.595702 2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 19:11:26.963230 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.595793 2556 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 19:11:26.963230 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.596796 2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 19:11:26.963230 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.600421 2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 19:11:26.963230 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.600844 2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-104.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 19:11:26.963230 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.602248 2556 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.602460 2556 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.602469 2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.603301 2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.604477 2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.607280 2556 state_mem.go:36] "Initialized new in-memory state store" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.607392 2556 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.607423 2556 kubelet.go:486] "Attempting to sync node with API server" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.607432 2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.607442 2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.607452 2556 kubelet.go:398] "Adding apiserver pod source" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.607461 2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.607885 2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.607903 2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.610265 2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.612193 2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.612221 2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.613303 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.613317 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.613325 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.613330 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.613343 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.613349 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.613361 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.613366 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.613377 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.613383 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.613391 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.613396 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.614141 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.614156 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.619173 2556 server.go:1268] "Started kubelet" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.620388 2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.620476 2556 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.620672 2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.621582 2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-104.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:26.621857 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-104.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:26.622117 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.623528 2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 19:11:26.964329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.624372 2556 server.go:310] "Adding debug handlers to kubelet server" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:26.628159 2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.629348 2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:26.628468 2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-104.ec2.internal.18be8ec6b7888e51 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-104.ec2.internal,UID:ip-10-0-132-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-104.ec2.internal,},FirstTimestamp:2026-07-02 19:11:26.619143761 +0000 UTC m=+0.351140432,LastTimestamp:2026-07-02 19:11:26.619143761 +0000 UTC m=+0.351140432,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-104.ec2.internal,}" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.629372 2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:26.629483 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-104.ec2.internal\" not found" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.629938 2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.629940 2556 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.629964 2556 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.630042 2556 reconstruct.go:97] "Volume reconstruction finished" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.630051 2556 reconciler.go:29] "Reconciler: start to sync state" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.631682 2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.631696 2556 factory.go:55] Registering systemd factory Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.631705 2556 factory.go:223] Registration of the systemd container factory successfully Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.631950 2556 factory.go:153] Registering CRI-O factory Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.631965 2556 factory.go:223] Registration of the crio container factory successfully Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.631984 2556 factory.go:103] Registering Raw factory Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.631999 2556 manager.go:1196] Started watching for new ooms in manager Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.632374 2556 manager.go:349] Starting recovery of all containers Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.639709 2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:26.641176 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:26.641215 2556 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-104.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.643988 2556 manager.go:354] Recovery completed Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.648227 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.650090 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.650122 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.650138 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.650640 2556 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.650651 2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.650655 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.650666 2556 state_mem.go:36] "Initialized new in-memory state store" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.651327 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.651351 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:11:26.965600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.651361 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.652641 2556 policy_none.go:49] "None policy: Start" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.652656 2556 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.652669 2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.654295 2556 policy_none.go:47] "Start" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.658575 2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hxlw7" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:26.659161 2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-104.ec2.internal.18be8ec6b9610654 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-104.ec2.internal,UID:ip-10-0-132-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-104.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-104.ec2.internal,},FirstTimestamp:2026-07-02 19:11:26.650107476 +0000 UTC m=+0.382104149,LastTimestamp:2026-07-02 19:11:26.650107476 +0000 UTC m=+0.382104149,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-104.ec2.internal,}" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:26.663777 2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-104.ec2.internal.18be8ec6b961518a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-104.ec2.internal,UID:ip-10-0-132-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-104.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-104.ec2.internal,},FirstTimestamp:2026-07-02 19:11:26.65012673 +0000 UTC m=+0.382123400,LastTimestamp:2026-07-02 19:11:26.65012673 +0000 UTC m=+0.382123400,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-104.ec2.internal,}" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.665319 2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hxlw7" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.719504 2556 manager.go:337] "Starting Device Plugin manager" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:26.719592 2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.719605 2556 server.go:87] "Starting device plugin registration server" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.719884 2556 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.719897 2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.720018 2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.720134 2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.720145 2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:26.720589 2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:26.720639 2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-104.ec2.internal\" not found" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.786455 2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.786703 2556 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.786731 2556 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:26.786774 2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.789385 2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.820577 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.821535 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.821571 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:11:27.064569 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.821587 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.821617 2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-104.ec2.internal" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.834007 2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-104.ec2.internal" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:26.834027 2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-104.ec2.internal\": node \"ip-10-0-132-104.ec2.internal\" not found" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:26.875340 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-104.ec2.internal\" not found" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.887380 2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-104.ec2.internal"] Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.887458 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.889340 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.889379 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.889394 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.890921 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.890998 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.891057 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.891564 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.891586 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.891595 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.891678 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.891713 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.891726 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.892931 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-104.ec2.internal" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.892954 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.893624 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.893646 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:26.893656 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:26.934813 2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-104.ec2.internal\" not found" node="ip-10-0-132-104.ec2.internal" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:26.939722 2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-104.ec2.internal\" not found" node="ip-10-0-132-104.ec2.internal" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:26.976290 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-104.ec2.internal\" not found" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:27.031280 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/21f1b9fd1a90843c87518bdaf702a316-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal\" (UID: \"21f1b9fd1a90843c87518bdaf702a316\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal" Jul 02 19:11:27.065610 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:27.031320 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21f1b9fd1a90843c87518bdaf702a316-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal\" (UID: \"21f1b9fd1a90843c87518bdaf702a316\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal" Jul 02 19:11:27.066368 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:27.031337 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d9ea7e94a50077843ac883931f2f632a-config\") pod \"kube-apiserver-proxy-ip-10-0-132-104.ec2.internal\" (UID: \"d9ea7e94a50077843ac883931f2f632a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-104.ec2.internal" Jul 02 19:11:27.171173 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:27.076508 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-104.ec2.internal\" not found" Jul 02 19:11:27.171173 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:27.132441 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/21f1b9fd1a90843c87518bdaf702a316-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal\" (UID: \"21f1b9fd1a90843c87518bdaf702a316\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal" Jul 02 19:11:27.171173 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:27.132478 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21f1b9fd1a90843c87518bdaf702a316-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal\" (UID: \"21f1b9fd1a90843c87518bdaf702a316\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal" Jul 02 19:11:27.171173 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:27.132500 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d9ea7e94a50077843ac883931f2f632a-config\") pod \"kube-apiserver-proxy-ip-10-0-132-104.ec2.internal\" (UID: \"d9ea7e94a50077843ac883931f2f632a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-104.ec2.internal" Jul 02 19:11:27.171173 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:27.132566 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d9ea7e94a50077843ac883931f2f632a-config\") pod \"kube-apiserver-proxy-ip-10-0-132-104.ec2.internal\" (UID: \"d9ea7e94a50077843ac883931f2f632a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-104.ec2.internal" Jul 02 19:11:27.171173 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:27.132631 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/21f1b9fd1a90843c87518bdaf702a316-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal\" (UID: \"21f1b9fd1a90843c87518bdaf702a316\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal" Jul 02 19:11:27.171173 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:27.132664 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21f1b9fd1a90843c87518bdaf702a316-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal\" (UID: \"21f1b9fd1a90843c87518bdaf702a316\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal" Jul 02 19:11:27.251023 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:27.177552 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-104.ec2.internal\" not found" Jul 02 19:11:27.251023 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:27.242676 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal" Jul 02 19:11:27.251023 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:27.243804 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-104.ec2.internal" Jul 02 19:11:27.358241 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:27.290573 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-104.ec2.internal\" not found" Jul 02 19:11:27.476994 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:27.391118 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-104.ec2.internal\" not found" Jul 02 19:11:27.561721 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:27.495715 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-104.ec2.internal\" not found" Jul 02 19:11:27.561721 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:27.514744 2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:11:27.561721 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:27.561001 2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 19:11:27.561721 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:27.561137 2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 19:11:27.561721 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:27.561161 2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 19:11:28.849460 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:27.596288 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-104.ec2.internal\" not found" Jul 02 19:11:28.852838 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:28.852156 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 19:06:27 +0000 UTC" deadline="2027-11-26 01:02:08.706851938 +0000 UTC" Jul 02 19:11:28.852838 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:28.852187 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12269h50m39.854668714s" Jul 02 19:11:28.854033 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:28.853797 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:11:28.948402 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:28.854941 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:11:28.948402 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:28.854970 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:11:28.948402 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:28.854979 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:11:28.948402 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:28.882416 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-104.ec2.internal\" not found" Jul 02 19:11:29.027497 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:28.983356 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-104.ec2.internal\" not found" Jul 02 19:11:29.106584 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:29.083935 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-104.ec2.internal\" not found" Jul 02 19:11:29.106584 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:29.097895 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f1b9fd1a90843c87518bdaf702a316.slice/crio-61ec38406733b9b5ecb2e38660e438f6ba1203c28134abcb3385d4d496bc5d6e WatchSource:0}: Error finding container 61ec38406733b9b5ecb2e38660e438f6ba1203c28134abcb3385d4d496bc5d6e: Status 404 returned error can't find the container with id 61ec38406733b9b5ecb2e38660e438f6ba1203c28134abcb3385d4d496bc5d6e Jul 02 19:11:29.106584 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:29.098272 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9ea7e94a50077843ac883931f2f632a.slice/crio-207b50a0c801d776a2c44279e005baf54290c27653edfd7c03cdb14b9e2af25a WatchSource:0}: Error finding container 207b50a0c801d776a2c44279e005baf54290c27653edfd7c03cdb14b9e2af25a: Status 404 returned error can't find the container with id 207b50a0c801d776a2c44279e005baf54290c27653edfd7c03cdb14b9e2af25a Jul 02 19:11:29.106584 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.104737 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 19:11:29.198838 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.136160 2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:11:29.330816 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.215736 2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-104.ec2.internal" Jul 02 19:11:29.330816 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.231696 2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 19:11:29.330816 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.232758 2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal" Jul 02 19:11:29.330816 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.233398 2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:11:29.330816 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.248365 2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 19:11:29.794773 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.794699 2556 apiserver.go:52] "Watching apiserver" Jul 02 19:11:29.803877 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.803850 2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 19:11:29.805952 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.805925 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-104.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl","openshift-cluster-node-tuning-operator/tuned-dwn48","openshift-dns/node-resolver-sv2qm","openshift-image-registry/node-ca-p8crb","openshift-multus/network-metrics-daemon-pjsd8","openshift-network-operator/iptables-alerter-5m2z2","openshift-ovn-kubernetes/ovnkube-node-p2j8h","kube-system/global-pull-secret-syncer-bhgfb","kube-system/konnectivity-agent-c5wmt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal","openshift-multus/multus-additional-cni-plugins-7skzd","openshift-multus/multus-fqqsq","openshift-network-diagnostics/network-check-target-q6q7s"] Jul 02 19:11:29.807550 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.807526 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5m2z2" Jul 02 19:11:29.808740 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.808718 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.810259 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.809861 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.811178 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.811141 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sv2qm" Jul 02 19:11:29.812623 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.812433 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p8crb" Jul 02 19:11:29.813786 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.813767 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 19:11:29.814018 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.814002 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 19:11:29.814374 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.814358 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fj26f\"" Jul 02 19:11:29.814627 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.814555 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 19:11:29.814627 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.814578 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:29.814761 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:29.814658 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:11:29.814761 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.814682 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rm88z\"" Jul 02 19:11:29.814761 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.814749 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 19:11:29.814905 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.814586 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 19:11:29.815015 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.814996 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 19:11:29.815108 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.815016 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 19:11:29.815179 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.815008 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 19:11:29.815876 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.815664 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6kpqz\"" Jul 02 19:11:29.815876 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.815819 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 19:11:29.816164 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.816030 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:29.816164 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:29.816115 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhgfb" podUID="882fce17-202b-44a2-9002-efdcb888f969" Jul 02 19:11:29.818309 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.818214 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:29.818309 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:29.818274 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6q7s" podUID="c8e6d112-aeda-4479-9158-3e91f551e95c" Jul 02 19:11:29.818540 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.818516 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 19:11:29.819092 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.818900 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 19:11:29.819092 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.818949 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 19:11:29.819294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.819095 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 19:11:29.819294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.819277 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 19:11:29.819412 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.819338 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rj4z5\"" Jul 02 19:11:29.820522 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.820503 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:29.820818 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.820751 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bsvxx\"" Jul 02 19:11:29.822899 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.822881 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c5wmt" Jul 02 19:11:29.824987 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.824968 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.826100 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.826081 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:29.826646 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.826498 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 19:11:29.826646 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.826579 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 19:11:29.826802 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.826702 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4cmmz\"" Jul 02 19:11:29.826857 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.826810 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 19:11:29.827876 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.827758 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 19:11:29.829796 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.829327 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 19:11:29.829796 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.829436 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 19:11:29.829796 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.829443 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pk76s\"" Jul 02 19:11:29.830006 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.829860 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 19:11:29.830064 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.830014 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 19:11:29.830200 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.830181 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-chpbq\"" Jul 02 19:11:29.830257 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.830190 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 19:11:29.830733 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.830715 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 19:11:29.832419 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.832403 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 19:11:29.832612 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.832595 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bpt8l\"" Jul 02 19:11:29.832789 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.832773 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 19:11:29.833065 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.833049 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 19:11:29.835409 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.835390 2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 19:11:29.840251 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840229 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-sysctl-d\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.840346 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840264 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-sysctl-conf\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.840346 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840287 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-run\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.840346 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840310 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4b69d5a7-876e-445e-b851-69b5e139d438-iptables-alerter-script\") pod \"iptables-alerter-5m2z2\" (UID: \"4b69d5a7-876e-445e-b851-69b5e139d438\") " pod="openshift-network-operator/iptables-alerter-5m2z2" Jul 02 19:11:29.840346 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840335 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx9qf\" (UniqueName: \"kubernetes.io/projected/4b69d5a7-876e-445e-b851-69b5e139d438-kube-api-access-dx9qf\") pod \"iptables-alerter-5m2z2\" (UID: \"4b69d5a7-876e-445e-b851-69b5e139d438\") " pod="openshift-network-operator/iptables-alerter-5m2z2" Jul 02 19:11:29.840549 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840357 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-host-var-lib-cni-bin\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.840549 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840379 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdkzm\" (UniqueName: \"kubernetes.io/projected/b650207c-2f59-4141-80f1-2cd88e81c5e9-kube-api-access-vdkzm\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.840549 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840402 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-multus-conf-dir\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.840549 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840425 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-host-run-multus-certs\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.840549 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840446 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/882fce17-202b-44a2-9002-efdcb888f969-kubelet-config\") pod \"global-pull-secret-syncer-bhgfb\" (UID: \"882fce17-202b-44a2-9002-efdcb888f969\") " pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:29.840549 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840468 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-host-var-lib-kubelet\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.840549 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840507 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-kubernetes\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.840807 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840552 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-host\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.840807 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840584 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b650207c-2f59-4141-80f1-2cd88e81c5e9-tmp\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.840807 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840611 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dee24e3b-8c4a-48d3-9c21-31c9ca499d72-hosts-file\") pod \"node-resolver-sv2qm\" (UID: \"dee24e3b-8c4a-48d3-9c21-31c9ca499d72\") " pod="openshift-dns/node-resolver-sv2qm" Jul 02 19:11:29.840807 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840640 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkjt9\" (UniqueName: \"kubernetes.io/projected/dee24e3b-8c4a-48d3-9c21-31c9ca499d72-kube-api-access-pkjt9\") pod \"node-resolver-sv2qm\" (UID: \"dee24e3b-8c4a-48d3-9c21-31c9ca499d72\") " pod="openshift-dns/node-resolver-sv2qm" Jul 02 19:11:29.840807 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840681 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-multus-cni-dir\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.840807 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840708 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-host-run-k8s-cni-cncf-io\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.840807 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840729 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-modprobe-d\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.840807 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840757 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dee24e3b-8c4a-48d3-9c21-31c9ca499d72-tmp-dir\") pod \"node-resolver-sv2qm\" (UID: \"dee24e3b-8c4a-48d3-9c21-31c9ca499d72\") " pod="openshift-dns/node-resolver-sv2qm" Jul 02 19:11:29.840807 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840778 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-system-cni-dir\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.840807 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840800 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d0af660-54fe-44aa-b75e-a0116a2f836d-cni-binary-copy\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.841294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840838 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-sysconfig\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.841294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840862 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-systemd\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.841294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840898 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-var-lib-kubelet\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.841294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840925 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-tuned\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.841294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840950 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x68k8\" (UniqueName: \"kubernetes.io/projected/205e2e72-bfab-43b6-b8d3-8af9a1c8561a-kube-api-access-x68k8\") pod \"node-ca-p8crb\" (UID: \"205e2e72-bfab-43b6-b8d3-8af9a1c8561a\") " pod="openshift-image-registry/node-ca-p8crb" Jul 02 19:11:29.841294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840973 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs\") pod \"network-metrics-daemon-pjsd8\" (UID: \"75464ead-f0fb-4ea8-beba-55b639087860\") " pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:29.841294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.840992 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-os-release\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.841294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.841023 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-host-run-netns\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.841294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.841044 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-etc-kubernetes\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.841294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.841058 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-sys\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.841294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.841071 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-lib-modules\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.841294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.841093 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/205e2e72-bfab-43b6-b8d3-8af9a1c8561a-serviceca\") pod \"node-ca-p8crb\" (UID: \"205e2e72-bfab-43b6-b8d3-8af9a1c8561a\") " pod="openshift-image-registry/node-ca-p8crb" Jul 02 19:11:29.841294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.841160 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret\") pod \"global-pull-secret-syncer-bhgfb\" (UID: \"882fce17-202b-44a2-9002-efdcb888f969\") " pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:29.841294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.841191 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-cnibin\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.841818 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.841216 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-multus-socket-dir-parent\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.841818 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.841240 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5d0af660-54fe-44aa-b75e-a0116a2f836d-multus-daemon-config\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.841818 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.841264 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/205e2e72-bfab-43b6-b8d3-8af9a1c8561a-host\") pod \"node-ca-p8crb\" (UID: \"205e2e72-bfab-43b6-b8d3-8af9a1c8561a\") " pod="openshift-image-registry/node-ca-p8crb" Jul 02 19:11:29.841818 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.841287 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8m54\" (UniqueName: \"kubernetes.io/projected/75464ead-f0fb-4ea8-beba-55b639087860-kube-api-access-p8m54\") pod \"network-metrics-daemon-pjsd8\" (UID: \"75464ead-f0fb-4ea8-beba-55b639087860\") " pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:29.841818 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.841310 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/882fce17-202b-44a2-9002-efdcb888f969-dbus\") pod \"global-pull-secret-syncer-bhgfb\" (UID: \"882fce17-202b-44a2-9002-efdcb888f969\") " pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:29.841818 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.841342 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b69d5a7-876e-445e-b851-69b5e139d438-host-slash\") pod \"iptables-alerter-5m2z2\" (UID: \"4b69d5a7-876e-445e-b851-69b5e139d438\") " pod="openshift-network-operator/iptables-alerter-5m2z2" Jul 02 19:11:29.841818 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.841376 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-host-var-lib-cni-multus\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.841818 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.841400 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-hostroot\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.841818 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.841422 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjk48\" (UniqueName: \"kubernetes.io/projected/5d0af660-54fe-44aa-b75e-a0116a2f836d-kube-api-access-bjk48\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.864390 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.864364 2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 19:11:29.893866 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.893841 2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-74cbr" Jul 02 19:11:29.902116 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.902093 2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-74cbr" Jul 02 19:11:29.916283 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.916263 2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 19:11:29.941712 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.941687 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-sysctl-d\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.941832 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.941722 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-run\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.941832 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.941746 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdkzm\" (UniqueName: \"kubernetes.io/projected/b650207c-2f59-4141-80f1-2cd88e81c5e9-kube-api-access-vdkzm\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.941832 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.941772 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.941832 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.941803 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-system-cni-dir\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:29.941832 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.941826 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n96g5\" (UniqueName: \"kubernetes.io/projected/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-kube-api-access-n96g5\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:29.942237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.941850 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4b69d5a7-876e-445e-b851-69b5e139d438-iptables-alerter-script\") pod \"iptables-alerter-5m2z2\" (UID: \"4b69d5a7-876e-445e-b851-69b5e139d438\") " pod="openshift-network-operator/iptables-alerter-5m2z2" Jul 02 19:11:29.942237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.941877 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx9qf\" (UniqueName: \"kubernetes.io/projected/4b69d5a7-876e-445e-b851-69b5e139d438-kube-api-access-dx9qf\") pod \"iptables-alerter-5m2z2\" (UID: \"4b69d5a7-876e-445e-b851-69b5e139d438\") " pod="openshift-network-operator/iptables-alerter-5m2z2" Jul 02 19:11:29.942237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.941913 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-cni-netd\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.942237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.941935 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-cnibin\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:29.942237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.941957 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:29.942237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.941981 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-host-run-multus-certs\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.942237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942006 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-run-openvswitch\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.942237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942029 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/882fce17-202b-44a2-9002-efdcb888f969-kubelet-config\") pod \"global-pull-secret-syncer-bhgfb\" (UID: \"882fce17-202b-44a2-9002-efdcb888f969\") " pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:29.942237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942082 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-host\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.942237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942108 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4a8d6b85-07f6-4d4c-86d4-3dadb68f48af-konnectivity-ca\") pod \"konnectivity-agent-c5wmt\" (UID: \"4a8d6b85-07f6-4d4c-86d4-3dadb68f48af\") " pod="kube-system/konnectivity-agent-c5wmt" Jul 02 19:11:29.942237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942134 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-host-run-k8s-cni-cncf-io\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.942237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942184 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-modprobe-d\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.942237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942212 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dee24e3b-8c4a-48d3-9c21-31c9ca499d72-tmp-dir\") pod \"node-resolver-sv2qm\" (UID: \"dee24e3b-8c4a-48d3-9c21-31c9ca499d72\") " pod="openshift-dns/node-resolver-sv2qm" Jul 02 19:11:29.942237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942236 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-system-cni-dir\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.943057 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942258 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-sysconfig\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.943057 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942282 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-systemd\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.943057 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942307 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-var-lib-kubelet\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.943057 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942332 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x68k8\" (UniqueName: \"kubernetes.io/projected/205e2e72-bfab-43b6-b8d3-8af9a1c8561a-kube-api-access-x68k8\") pod \"node-ca-p8crb\" (UID: \"205e2e72-bfab-43b6-b8d3-8af9a1c8561a\") " pod="openshift-image-registry/node-ca-p8crb" Jul 02 19:11:29.943057 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942357 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs\") pod \"network-metrics-daemon-pjsd8\" (UID: \"75464ead-f0fb-4ea8-beba-55b639087860\") " pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:29.943057 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942382 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/263d1a36-f35e-47e0-b7b4-696f5ebc8621-ovn-node-metrics-cert\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.943057 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942407 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/263d1a36-f35e-47e0-b7b4-696f5ebc8621-ovnkube-script-lib\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.943057 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942431 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-etc-kubernetes\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.943057 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942457 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/205e2e72-bfab-43b6-b8d3-8af9a1c8561a-serviceca\") pod \"node-ca-p8crb\" (UID: \"205e2e72-bfab-43b6-b8d3-8af9a1c8561a\") " pod="openshift-image-registry/node-ca-p8crb" Jul 02 19:11:29.943057 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942481 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2gss\" (UniqueName: \"kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss\") pod \"network-check-target-q6q7s\" (UID: \"c8e6d112-aeda-4479-9158-3e91f551e95c\") " pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:29.943057 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942509 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/263d1a36-f35e-47e0-b7b4-696f5ebc8621-ovnkube-config\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.943057 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942535 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-os-release\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:29.943057 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942561 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:29.943057 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942584 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/92464f4c-5503-4d92-80b5-053bd4f3f041-sys-fs\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:29.943713 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942611 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret\") pod \"global-pull-secret-syncer-bhgfb\" (UID: \"882fce17-202b-44a2-9002-efdcb888f969\") " pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:29.943713 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942635 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5d0af660-54fe-44aa-b75e-a0116a2f836d-multus-daemon-config\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.943713 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942660 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/205e2e72-bfab-43b6-b8d3-8af9a1c8561a-host\") pod \"node-ca-p8crb\" (UID: \"205e2e72-bfab-43b6-b8d3-8af9a1c8561a\") " pod="openshift-image-registry/node-ca-p8crb" Jul 02 19:11:29.943713 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942684 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8m54\" (UniqueName: \"kubernetes.io/projected/75464ead-f0fb-4ea8-beba-55b639087860-kube-api-access-p8m54\") pod \"network-metrics-daemon-pjsd8\" (UID: \"75464ead-f0fb-4ea8-beba-55b639087860\") " pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:29.943713 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942713 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-systemd-units\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.943713 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942737 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-run-systemd\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.943713 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942776 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-cni-bin\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.943713 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942800 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2bdc\" (UniqueName: \"kubernetes.io/projected/263d1a36-f35e-47e0-b7b4-696f5ebc8621-kube-api-access-f2bdc\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.943713 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942839 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/882fce17-202b-44a2-9002-efdcb888f969-dbus\") pod \"global-pull-secret-syncer-bhgfb\" (UID: \"882fce17-202b-44a2-9002-efdcb888f969\") " pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:29.943713 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942867 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b69d5a7-876e-445e-b851-69b5e139d438-host-slash\") pod \"iptables-alerter-5m2z2\" (UID: \"4b69d5a7-876e-445e-b851-69b5e139d438\") " pod="openshift-network-operator/iptables-alerter-5m2z2" Jul 02 19:11:29.943713 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942892 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-host-var-lib-cni-multus\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.943713 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942916 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-hostroot\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.943713 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942942 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjk48\" (UniqueName: \"kubernetes.io/projected/5d0af660-54fe-44aa-b75e-a0116a2f836d-kube-api-access-bjk48\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.943713 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.942966 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-sysctl-conf\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.944464 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943001 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:29.944464 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943026 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/92464f4c-5503-4d92-80b5-053bd4f3f041-registration-dir\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:29.944464 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943051 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-host-var-lib-cni-bin\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.944464 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943085 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-log-socket\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.944464 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943111 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-multus-conf-dir\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.944464 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943135 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-run-netns\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.944464 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943177 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-node-log\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.944464 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943202 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/92464f4c-5503-4d92-80b5-053bd4f3f041-device-dir\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:29.944464 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943227 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9fxf\" (UniqueName: \"kubernetes.io/projected/92464f4c-5503-4d92-80b5-053bd4f3f041-kube-api-access-m9fxf\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:29.944464 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943255 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-host-var-lib-kubelet\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.944464 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943280 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-kubernetes\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.944464 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943307 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b650207c-2f59-4141-80f1-2cd88e81c5e9-tmp\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.944464 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943333 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dee24e3b-8c4a-48d3-9c21-31c9ca499d72-hosts-file\") pod \"node-resolver-sv2qm\" (UID: \"dee24e3b-8c4a-48d3-9c21-31c9ca499d72\") " pod="openshift-dns/node-resolver-sv2qm" Jul 02 19:11:29.944464 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943363 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkjt9\" (UniqueName: \"kubernetes.io/projected/dee24e3b-8c4a-48d3-9c21-31c9ca499d72-kube-api-access-pkjt9\") pod \"node-resolver-sv2qm\" (UID: \"dee24e3b-8c4a-48d3-9c21-31c9ca499d72\") " pod="openshift-dns/node-resolver-sv2qm" Jul 02 19:11:29.945217 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943389 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-kubelet\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.945217 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943412 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-run-ovn\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.945217 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943448 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-cni-binary-copy\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:29.945217 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943478 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-multus-cni-dir\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.945217 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943502 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-slash\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.945217 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943529 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d0af660-54fe-44aa-b75e-a0116a2f836d-cni-binary-copy\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.945217 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943557 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-tuned\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.945217 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943583 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-var-lib-openvswitch\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.945217 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943609 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/263d1a36-f35e-47e0-b7b4-696f5ebc8621-env-overrides\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.945217 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943634 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/92464f4c-5503-4d92-80b5-053bd4f3f041-socket-dir\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:29.945217 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943662 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4a8d6b85-07f6-4d4c-86d4-3dadb68f48af-agent-certs\") pod \"konnectivity-agent-c5wmt\" (UID: \"4a8d6b85-07f6-4d4c-86d4-3dadb68f48af\") " pod="kube-system/konnectivity-agent-c5wmt" Jul 02 19:11:29.945217 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943690 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-os-release\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.945217 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943717 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-host-run-netns\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.945217 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943741 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-sys\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.946024 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943768 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-lib-modules\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.946024 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943792 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-run-ovn-kubernetes\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.946024 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943822 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/92464f4c-5503-4d92-80b5-053bd4f3f041-etc-selinux\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:29.946024 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943854 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-cnibin\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.946024 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943881 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-multus-socket-dir-parent\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.946024 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943906 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-etc-openvswitch\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:29.946024 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.943930 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/92464f4c-5503-4d92-80b5-053bd4f3f041-kubelet-dir\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:29.946024 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.944129 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-sysctl-d\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.946024 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.944253 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-run\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.946024 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.945066 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4b69d5a7-876e-445e-b851-69b5e139d438-iptables-alerter-script\") pod \"iptables-alerter-5m2z2\" (UID: \"4b69d5a7-876e-445e-b851-69b5e139d438\") " pod="openshift-network-operator/iptables-alerter-5m2z2" Jul 02 19:11:29.946024 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.945437 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-host-run-multus-certs\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.946024 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.945507 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/882fce17-202b-44a2-9002-efdcb888f969-kubelet-config\") pod \"global-pull-secret-syncer-bhgfb\" (UID: \"882fce17-202b-44a2-9002-efdcb888f969\") " pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:29.946024 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.945559 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-host\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.946024 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.945617 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-host-run-k8s-cni-cncf-io\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.946736 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.945717 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-modprobe-d\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.946736 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.945972 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dee24e3b-8c4a-48d3-9c21-31c9ca499d72-tmp-dir\") pod \"node-resolver-sv2qm\" (UID: \"dee24e3b-8c4a-48d3-9c21-31c9ca499d72\") " pod="openshift-dns/node-resolver-sv2qm" Jul 02 19:11:29.946736 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.946086 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-system-cni-dir\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.946736 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.946141 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-sysconfig\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.946736 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.946205 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-systemd\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.946736 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.946255 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-var-lib-kubelet\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.946736 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:29.946532 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:11:29.946736 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:29.946594 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs podName:75464ead-f0fb-4ea8-beba-55b639087860 nodeName:}" failed. No retries permitted until 2026-07-02 19:11:30.446573601 +0000 UTC m=+2.993298605 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs") pod "network-metrics-daemon-pjsd8" (UID: "75464ead-f0fb-4ea8-beba-55b639087860") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:11:29.947186 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.946799 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-etc-kubernetes\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.947245 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.947212 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/205e2e72-bfab-43b6-b8d3-8af9a1c8561a-serviceca\") pod \"node-ca-p8crb\" (UID: \"205e2e72-bfab-43b6-b8d3-8af9a1c8561a\") " pod="openshift-image-registry/node-ca-p8crb" Jul 02 19:11:29.947806 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:29.947343 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:11:29.947806 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:29.947391 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret podName:882fce17-202b-44a2-9002-efdcb888f969 nodeName:}" failed. No retries permitted until 2026-07-02 19:11:30.447377741 +0000 UTC m=+2.994102738 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret") pod "global-pull-secret-syncer-bhgfb" (UID: "882fce17-202b-44a2-9002-efdcb888f969") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:11:29.955976 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.953030 2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 19:11:29.955976 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.953074 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5d0af660-54fe-44aa-b75e-a0116a2f836d-multus-daemon-config\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.955976 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.953219 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dee24e3b-8c4a-48d3-9c21-31c9ca499d72-hosts-file\") pod \"node-resolver-sv2qm\" (UID: \"dee24e3b-8c4a-48d3-9c21-31c9ca499d72\") " pod="openshift-dns/node-resolver-sv2qm" Jul 02 19:11:29.955976 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.953600 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-multus-cni-dir\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.955976 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.953540 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-sysctl-conf\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.955976 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.954022 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/205e2e72-bfab-43b6-b8d3-8af9a1c8561a-host\") pod \"node-ca-p8crb\" (UID: \"205e2e72-bfab-43b6-b8d3-8af9a1c8561a\") " pod="openshift-image-registry/node-ca-p8crb" Jul 02 19:11:29.955976 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.954099 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d0af660-54fe-44aa-b75e-a0116a2f836d-cni-binary-copy\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.955976 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.954395 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/882fce17-202b-44a2-9002-efdcb888f969-dbus\") pod \"global-pull-secret-syncer-bhgfb\" (UID: \"882fce17-202b-44a2-9002-efdcb888f969\") " pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:29.955976 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.954463 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-host-var-lib-cni-multus\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.955976 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.954481 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-os-release\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.955976 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.954515 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-hostroot\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.955976 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.954550 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b69d5a7-876e-445e-b851-69b5e139d438-host-slash\") pod \"iptables-alerter-5m2z2\" (UID: \"4b69d5a7-876e-445e-b851-69b5e139d438\") " pod="openshift-network-operator/iptables-alerter-5m2z2" Jul 02 19:11:29.955976 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.954605 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-host-run-netns\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.955976 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.954663 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-sys\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.955976 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.954769 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-lib-modules\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.955976 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.954989 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-multus-socket-dir-parent\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.956808 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.955060 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-cnibin\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.956808 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.955170 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-host-var-lib-cni-bin\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.956808 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.955235 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-multus-conf-dir\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.956808 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.955325 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d0af660-54fe-44aa-b75e-a0116a2f836d-host-var-lib-kubelet\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.956808 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.955373 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-kubernetes\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.962099 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.962008 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkjt9\" (UniqueName: \"kubernetes.io/projected/dee24e3b-8c4a-48d3-9c21-31c9ca499d72-kube-api-access-pkjt9\") pod \"node-resolver-sv2qm\" (UID: \"dee24e3b-8c4a-48d3-9c21-31c9ca499d72\") " pod="openshift-dns/node-resolver-sv2qm" Jul 02 19:11:29.962908 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.962877 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjk48\" (UniqueName: \"kubernetes.io/projected/5d0af660-54fe-44aa-b75e-a0116a2f836d-kube-api-access-bjk48\") pod \"multus-fqqsq\" (UID: \"5d0af660-54fe-44aa-b75e-a0116a2f836d\") " pod="openshift-multus/multus-fqqsq" Jul 02 19:11:29.964915 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.964792 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdkzm\" (UniqueName: \"kubernetes.io/projected/b650207c-2f59-4141-80f1-2cd88e81c5e9-kube-api-access-vdkzm\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.964915 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.964875 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8m54\" (UniqueName: \"kubernetes.io/projected/75464ead-f0fb-4ea8-beba-55b639087860-kube-api-access-p8m54\") pod \"network-metrics-daemon-pjsd8\" (UID: \"75464ead-f0fb-4ea8-beba-55b639087860\") " pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:29.965061 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.965003 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b650207c-2f59-4141-80f1-2cd88e81c5e9-etc-tuned\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.965931 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.965611 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx9qf\" (UniqueName: \"kubernetes.io/projected/4b69d5a7-876e-445e-b851-69b5e139d438-kube-api-access-dx9qf\") pod \"iptables-alerter-5m2z2\" (UID: \"4b69d5a7-876e-445e-b851-69b5e139d438\") " pod="openshift-network-operator/iptables-alerter-5m2z2" Jul 02 19:11:29.965931 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.965737 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b650207c-2f59-4141-80f1-2cd88e81c5e9-tmp\") pod \"tuned-dwn48\" (UID: \"b650207c-2f59-4141-80f1-2cd88e81c5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:29.967132 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.967088 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x68k8\" (UniqueName: \"kubernetes.io/projected/205e2e72-bfab-43b6-b8d3-8af9a1c8561a-kube-api-access-x68k8\") pod \"node-ca-p8crb\" (UID: \"205e2e72-bfab-43b6-b8d3-8af9a1c8561a\") " pod="openshift-image-registry/node-ca-p8crb" Jul 02 19:11:29.977703 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.977643 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal" event={"ID":"21f1b9fd1a90843c87518bdaf702a316","Type":"ContainerStarted","Data":"61ec38406733b9b5ecb2e38660e438f6ba1203c28134abcb3385d4d496bc5d6e"} Jul 02 19:11:29.979822 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:29.979773 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-104.ec2.internal" event={"ID":"d9ea7e94a50077843ac883931f2f632a","Type":"ContainerStarted","Data":"207b50a0c801d776a2c44279e005baf54290c27653edfd7c03cdb14b9e2af25a"} Jul 02 19:11:30.045074 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.044990 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-cni-netd\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.045074 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.045036 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-cnibin\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:30.045074 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.045063 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:30.045370 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.045090 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-run-openvswitch\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.045370 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.045123 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4a8d6b85-07f6-4d4c-86d4-3dadb68f48af-konnectivity-ca\") pod \"konnectivity-agent-c5wmt\" (UID: \"4a8d6b85-07f6-4d4c-86d4-3dadb68f48af\") " pod="kube-system/konnectivity-agent-c5wmt" Jul 02 19:11:30.045370 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.045190 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/263d1a36-f35e-47e0-b7b4-696f5ebc8621-ovn-node-metrics-cert\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.045370 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.045215 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/263d1a36-f35e-47e0-b7b4-696f5ebc8621-ovnkube-script-lib\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.045370 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.045242 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2gss\" (UniqueName: \"kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss\") pod \"network-check-target-q6q7s\" (UID: \"c8e6d112-aeda-4479-9158-3e91f551e95c\") " pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:30.045370 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.045266 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/263d1a36-f35e-47e0-b7b4-696f5ebc8621-ovnkube-config\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.045370 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.045288 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-os-release\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:30.045370 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.045312 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:30.045370 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.045342 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/92464f4c-5503-4d92-80b5-053bd4f3f041-sys-fs\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:30.045718 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.045377 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-systemd-units\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.045718 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.045399 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-run-systemd\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.045718 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.045420 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-cni-bin\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.045718 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.045441 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2bdc\" (UniqueName: \"kubernetes.io/projected/263d1a36-f35e-47e0-b7b4-696f5ebc8621-kube-api-access-f2bdc\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.045718 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.045470 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:30.045718 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.045605 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-run-openvswitch\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.046028 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046006 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:30.046064 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046027 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:30.046098 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046074 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-cni-netd\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.046098 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046075 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/92464f4c-5503-4d92-80b5-053bd4f3f041-registration-dir\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:30.046202 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046118 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-cnibin\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:30.046202 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046120 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-log-socket\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.046202 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046168 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4a8d6b85-07f6-4d4c-86d4-3dadb68f48af-konnectivity-ca\") pod \"konnectivity-agent-c5wmt\" (UID: \"4a8d6b85-07f6-4d4c-86d4-3dadb68f48af\") " pod="kube-system/konnectivity-agent-c5wmt" Jul 02 19:11:30.046295 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046201 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/92464f4c-5503-4d92-80b5-053bd4f3f041-sys-fs\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:30.046295 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046217 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-run-netns\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.046295 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046171 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-run-netns\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.046295 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046259 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-node-log\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.046295 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046284 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/92464f4c-5503-4d92-80b5-053bd4f3f041-device-dir\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:30.046440 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046308 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9fxf\" (UniqueName: \"kubernetes.io/projected/92464f4c-5503-4d92-80b5-053bd4f3f041-kube-api-access-m9fxf\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:30.046440 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046340 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-kubelet\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.046440 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046364 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-run-ovn\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.046440 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046387 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-cni-binary-copy\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:30.046440 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046395 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-systemd-units\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.046584 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046458 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-node-log\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.046584 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046460 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-slash\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.046584 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046498 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-run-systemd\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.046584 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046512 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/92464f4c-5503-4d92-80b5-053bd4f3f041-device-dir\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:30.046584 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046556 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-cni-bin\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.046736 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046710 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/263d1a36-f35e-47e0-b7b4-696f5ebc8621-ovnkube-script-lib\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.046803 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046751 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/92464f4c-5503-4d92-80b5-053bd4f3f041-registration-dir\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:30.046833 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046800 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-log-socket\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.046833 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046416 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-slash\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.046917 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046866 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-var-lib-openvswitch\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.046917 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046895 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/263d1a36-f35e-47e0-b7b4-696f5ebc8621-env-overrides\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.047006 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046919 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/92464f4c-5503-4d92-80b5-053bd4f3f041-socket-dir\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:30.047006 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046926 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-os-release\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:30.047006 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046948 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4a8d6b85-07f6-4d4c-86d4-3dadb68f48af-agent-certs\") pod \"konnectivity-agent-c5wmt\" (UID: \"4a8d6b85-07f6-4d4c-86d4-3dadb68f48af\") " pod="kube-system/konnectivity-agent-c5wmt" Jul 02 19:11:30.047006 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.046997 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-run-ovn-kubernetes\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.047207 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047010 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-var-lib-openvswitch\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.047207 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047022 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/92464f4c-5503-4d92-80b5-053bd4f3f041-etc-selinux\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:30.047207 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047026 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:30.047207 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047049 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-etc-openvswitch\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.047207 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047069 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-run-ovn\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.047207 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047074 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/92464f4c-5503-4d92-80b5-053bd4f3f041-kubelet-dir\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:30.047207 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047081 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-kubelet\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.047207 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047105 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-run-ovn-kubernetes\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.047207 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047103 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.047207 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047166 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.047683 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047216 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/92464f4c-5503-4d92-80b5-053bd4f3f041-socket-dir\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:30.047683 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047277 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/92464f4c-5503-4d92-80b5-053bd4f3f041-etc-selinux\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:30.047683 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047286 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/263d1a36-f35e-47e0-b7b4-696f5ebc8621-etc-openvswitch\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.047683 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047325 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/92464f4c-5503-4d92-80b5-053bd4f3f041-kubelet-dir\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:30.047683 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047420 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-system-cni-dir\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:30.047683 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047449 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n96g5\" (UniqueName: \"kubernetes.io/projected/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-kube-api-access-n96g5\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:30.047953 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047464 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/263d1a36-f35e-47e0-b7b4-696f5ebc8621-ovnkube-config\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.047953 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047488 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-system-cni-dir\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:30.047953 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047547 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-cni-binary-copy\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:30.047953 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.047863 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/263d1a36-f35e-47e0-b7b4-696f5ebc8621-env-overrides\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.050269 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.049797 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/263d1a36-f35e-47e0-b7b4-696f5ebc8621-ovn-node-metrics-cert\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.050269 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.049828 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4a8d6b85-07f6-4d4c-86d4-3dadb68f48af-agent-certs\") pod \"konnectivity-agent-c5wmt\" (UID: \"4a8d6b85-07f6-4d4c-86d4-3dadb68f48af\") " pod="kube-system/konnectivity-agent-c5wmt" Jul 02 19:11:30.056730 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:30.056706 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:11:30.056842 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:30.056733 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:11:30.056842 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:30.056747 2556 projected.go:196] Error preparing data for projected volume kube-api-access-d2gss for pod openshift-network-diagnostics/network-check-target-q6q7s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:11:30.056842 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:30.056828 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss podName:c8e6d112-aeda-4479-9158-3e91f551e95c nodeName:}" failed. No retries permitted until 2026-07-02 19:11:30.556807838 +0000 UTC m=+3.103532840 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-d2gss" (UniqueName: "kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss") pod "network-check-target-q6q7s" (UID: "c8e6d112-aeda-4479-9158-3e91f551e95c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:11:30.059805 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.059766 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2bdc\" (UniqueName: \"kubernetes.io/projected/263d1a36-f35e-47e0-b7b4-696f5ebc8621-kube-api-access-f2bdc\") pod \"ovnkube-node-p2j8h\" (UID: \"263d1a36-f35e-47e0-b7b4-696f5ebc8621\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.059999 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.059979 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9fxf\" (UniqueName: \"kubernetes.io/projected/92464f4c-5503-4d92-80b5-053bd4f3f041-kube-api-access-m9fxf\") pod \"aws-ebs-csi-driver-node-92zcl\" (UID: \"92464f4c-5503-4d92-80b5-053bd4f3f041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:30.060093 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.060067 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n96g5\" (UniqueName: \"kubernetes.io/projected/254f17a4-cbdb-420e-9fa7-d2a49ac7c026-kube-api-access-n96g5\") pod \"multus-additional-cni-plugins-7skzd\" (UID: \"254f17a4-cbdb-420e-9fa7-d2a49ac7c026\") " pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:30.122486 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.122457 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5m2z2" Jul 02 19:11:30.131387 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.131359 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fqqsq" Jul 02 19:11:30.131564 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:30.131535 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b69d5a7_876e_445e_b851_69b5e139d438.slice/crio-5c10eba42a1083cc72eef98a24c82236a0fe1093819a5048b69f22d34ab6185e WatchSource:0}: Error finding container 5c10eba42a1083cc72eef98a24c82236a0fe1093819a5048b69f22d34ab6185e: Status 404 returned error can't find the container with id 5c10eba42a1083cc72eef98a24c82236a0fe1093819a5048b69f22d34ab6185e Jul 02 19:11:30.140427 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:30.140404 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d0af660_54fe_44aa_b75e_a0116a2f836d.slice/crio-6349d538f67955bb365287a7e840204c6b23485f3a98ffdf95e65da170776984 WatchSource:0}: Error finding container 6349d538f67955bb365287a7e840204c6b23485f3a98ffdf95e65da170776984: Status 404 returned error can't find the container with id 6349d538f67955bb365287a7e840204c6b23485f3a98ffdf95e65da170776984 Jul 02 19:11:30.141725 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.141653 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c5wmt" Jul 02 19:11:30.147157 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.147125 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dwn48" Jul 02 19:11:30.154831 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.154815 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sv2qm" Jul 02 19:11:30.162419 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.162398 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p8crb" Jul 02 19:11:30.170990 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.170967 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" Jul 02 19:11:30.177756 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.177738 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:30.183528 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.183510 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7skzd" Jul 02 19:11:30.247887 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.247851 2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:11:30.450122 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.450088 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs\") pod \"network-metrics-daemon-pjsd8\" (UID: \"75464ead-f0fb-4ea8-beba-55b639087860\") " pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:30.450323 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.450162 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret\") pod \"global-pull-secret-syncer-bhgfb\" (UID: \"882fce17-202b-44a2-9002-efdcb888f969\") " pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:30.450323 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:30.450222 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:11:30.450323 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:30.450294 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs podName:75464ead-f0fb-4ea8-beba-55b639087860 nodeName:}" failed. No retries permitted until 2026-07-02 19:11:31.450275867 +0000 UTC m=+3.997000860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs") pod "network-metrics-daemon-pjsd8" (UID: "75464ead-f0fb-4ea8-beba-55b639087860") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:11:30.450323 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:30.450313 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:11:30.450540 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:30.450370 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret podName:882fce17-202b-44a2-9002-efdcb888f969 nodeName:}" failed. No retries permitted until 2026-07-02 19:11:31.450358419 +0000 UTC m=+3.997083404 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret") pod "global-pull-secret-syncer-bhgfb" (UID: "882fce17-202b-44a2-9002-efdcb888f969") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:11:30.651876 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.651834 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2gss\" (UniqueName: \"kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss\") pod \"network-check-target-q6q7s\" (UID: \"c8e6d112-aeda-4479-9158-3e91f551e95c\") " pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:30.652029 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:30.651991 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:11:30.652029 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:30.652017 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:11:30.652029 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:30.652031 2556 projected.go:196] Error preparing data for projected volume kube-api-access-d2gss for pod openshift-network-diagnostics/network-check-target-q6q7s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:11:30.652192 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:30.652099 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss podName:c8e6d112-aeda-4479-9158-3e91f551e95c nodeName:}" failed. No retries permitted until 2026-07-02 19:11:31.652079841 +0000 UTC m=+4.198804834 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-d2gss" (UniqueName: "kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss") pod "network-check-target-q6q7s" (UID: "c8e6d112-aeda-4479-9158-3e91f551e95c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:11:30.904458 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.904342 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 19:06:29 +0000 UTC" deadline="2028-02-06 02:04:23.960329966 +0000 UTC" Jul 02 19:11:30.904458 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.904409 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13998h52m53.055924494s" Jul 02 19:11:30.982334 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.982292 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5m2z2" event={"ID":"4b69d5a7-876e-445e-b851-69b5e139d438","Type":"ContainerStarted","Data":"5c10eba42a1083cc72eef98a24c82236a0fe1093819a5048b69f22d34ab6185e"} Jul 02 19:11:30.983383 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:30.983358 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fqqsq" event={"ID":"5d0af660-54fe-44aa-b75e-a0116a2f836d","Type":"ContainerStarted","Data":"6349d538f67955bb365287a7e840204c6b23485f3a98ffdf95e65da170776984"} Jul 02 19:11:31.119194 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:31.119128 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a8d6b85_07f6_4d4c_86d4_3dadb68f48af.slice/crio-ee8bf3ef150fe980021b5ec402f901853e970954ac7bea20104f1e4c8f106cf0 WatchSource:0}: Error finding container ee8bf3ef150fe980021b5ec402f901853e970954ac7bea20104f1e4c8f106cf0: Status 404 returned error can't find the container with id ee8bf3ef150fe980021b5ec402f901853e970954ac7bea20104f1e4c8f106cf0 Jul 02 19:11:31.120118 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:31.120089 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb650207c_2f59_4141_80f1_2cd88e81c5e9.slice/crio-5ede73543e35a2661fead38f6cb5860428c8cded914ac5099900f806b05bd293 WatchSource:0}: Error finding container 5ede73543e35a2661fead38f6cb5860428c8cded914ac5099900f806b05bd293: Status 404 returned error can't find the container with id 5ede73543e35a2661fead38f6cb5860428c8cded914ac5099900f806b05bd293 Jul 02 19:11:31.121287 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:31.121246 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod254f17a4_cbdb_420e_9fa7_d2a49ac7c026.slice/crio-5d0ff39475d31b0fa4de1f06edf18912e949862598e571f4a431382ce988a64b WatchSource:0}: Error finding container 5d0ff39475d31b0fa4de1f06edf18912e949862598e571f4a431382ce988a64b: Status 404 returned error can't find the container with id 5d0ff39475d31b0fa4de1f06edf18912e949862598e571f4a431382ce988a64b Jul 02 19:11:31.123763 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:31.123582 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod263d1a36_f35e_47e0_b7b4_696f5ebc8621.slice/crio-5df587a48957dff56793859bfcc120fa00048d1b130551a8e78d0e654e5175a3 WatchSource:0}: Error finding container 5df587a48957dff56793859bfcc120fa00048d1b130551a8e78d0e654e5175a3: Status 404 returned error can't find the container with id 5df587a48957dff56793859bfcc120fa00048d1b130551a8e78d0e654e5175a3 Jul 02 19:11:31.125075 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:31.125046 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod205e2e72_bfab_43b6_b8d3_8af9a1c8561a.slice/crio-87e974a638fd1cb898dc10552778c932060c19cb78d0181509ee31154f385ab5 WatchSource:0}: Error finding container 87e974a638fd1cb898dc10552778c932060c19cb78d0181509ee31154f385ab5: Status 404 returned error can't find the container with id 87e974a638fd1cb898dc10552778c932060c19cb78d0181509ee31154f385ab5 Jul 02 19:11:31.126630 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:31.126608 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92464f4c_5503_4d92_80b5_053bd4f3f041.slice/crio-b2807416ba159a19dbd62d8da5558c0cd07be002fa6adb9e4b8cb20c065cbef3 WatchSource:0}: Error finding container b2807416ba159a19dbd62d8da5558c0cd07be002fa6adb9e4b8cb20c065cbef3: Status 404 returned error can't find the container with id b2807416ba159a19dbd62d8da5558c0cd07be002fa6adb9e4b8cb20c065cbef3 Jul 02 19:11:31.128530 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:11:31.128480 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddee24e3b_8c4a_48d3_9c21_31c9ca499d72.slice/crio-8f0f1a5c0ec680161cf90476e834bf8b8011de88c4a1a6e0c2aaa94efd6dea67 WatchSource:0}: Error finding container 8f0f1a5c0ec680161cf90476e834bf8b8011de88c4a1a6e0c2aaa94efd6dea67: Status 404 returned error can't find the container with id 8f0f1a5c0ec680161cf90476e834bf8b8011de88c4a1a6e0c2aaa94efd6dea67 Jul 02 19:11:31.376016 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:31.375769 2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:11:31.459362 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:31.459211 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs\") pod \"network-metrics-daemon-pjsd8\" (UID: \"75464ead-f0fb-4ea8-beba-55b639087860\") " pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:31.459362 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:31.459277 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret\") pod \"global-pull-secret-syncer-bhgfb\" (UID: \"882fce17-202b-44a2-9002-efdcb888f969\") " pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:31.459561 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:31.459412 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:11:31.459561 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:31.459470 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret podName:882fce17-202b-44a2-9002-efdcb888f969 nodeName:}" failed. No retries permitted until 2026-07-02 19:11:33.459452498 +0000 UTC m=+6.006177489 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret") pod "global-pull-secret-syncer-bhgfb" (UID: "882fce17-202b-44a2-9002-efdcb888f969") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:11:31.459774 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:31.459751 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:11:31.459898 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:31.459820 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs podName:75464ead-f0fb-4ea8-beba-55b639087860 nodeName:}" failed. No retries permitted until 2026-07-02 19:11:33.459803509 +0000 UTC m=+6.006528502 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs") pod "network-metrics-daemon-pjsd8" (UID: "75464ead-f0fb-4ea8-beba-55b639087860") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:11:31.661980 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:31.661907 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2gss\" (UniqueName: \"kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss\") pod \"network-check-target-q6q7s\" (UID: \"c8e6d112-aeda-4479-9158-3e91f551e95c\") " pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:31.662163 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:31.662069 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:11:31.662163 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:31.662092 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:11:31.662163 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:31.662104 2556 projected.go:196] Error preparing data for projected volume kube-api-access-d2gss for pod openshift-network-diagnostics/network-check-target-q6q7s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:11:31.662340 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:31.662173 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss podName:c8e6d112-aeda-4479-9158-3e91f551e95c nodeName:}" failed. No retries permitted until 2026-07-02 19:11:33.662154371 +0000 UTC m=+6.208879371 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-d2gss" (UniqueName: "kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss") pod "network-check-target-q6q7s" (UID: "c8e6d112-aeda-4479-9158-3e91f551e95c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:11:31.905377 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:31.905284 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 19:06:29 +0000 UTC" deadline="2028-02-22 05:52:24.905045824 +0000 UTC" Jul 02 19:11:31.905377 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:31.905324 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14386h40m52.99972598s" Jul 02 19:11:31.973293 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:31.973264 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:31.973461 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:31.973410 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:11:31.974317 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:31.973809 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:31.974317 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:31.973902 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhgfb" podUID="882fce17-202b-44a2-9002-efdcb888f969" Jul 02 19:11:31.975857 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:31.975835 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:31.975953 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:31.975935 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6q7s" podUID="c8e6d112-aeda-4479-9158-3e91f551e95c" Jul 02 19:11:31.995523 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:31.995418 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" event={"ID":"263d1a36-f35e-47e0-b7b4-696f5ebc8621","Type":"ContainerStarted","Data":"5df587a48957dff56793859bfcc120fa00048d1b130551a8e78d0e654e5175a3"} Jul 02 19:11:32.004219 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:32.004181 2556 generic.go:356] "Generic (PLEG): container finished" podID="21f1b9fd1a90843c87518bdaf702a316" containerID="a450de217f948bca56decad3002ee3b176233533484ea606cadbf2bb01aefac9" exitCode=0 Jul 02 19:11:32.004555 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:32.004520 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal" event={"ID":"21f1b9fd1a90843c87518bdaf702a316","Type":"ContainerDied","Data":"a450de217f948bca56decad3002ee3b176233533484ea606cadbf2bb01aefac9"} Jul 02 19:11:32.009291 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:32.009214 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" event={"ID":"92464f4c-5503-4d92-80b5-053bd4f3f041","Type":"ContainerStarted","Data":"b2807416ba159a19dbd62d8da5558c0cd07be002fa6adb9e4b8cb20c065cbef3"} Jul 02 19:11:32.014297 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:32.014256 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7skzd" event={"ID":"254f17a4-cbdb-420e-9fa7-d2a49ac7c026","Type":"ContainerStarted","Data":"5d0ff39475d31b0fa4de1f06edf18912e949862598e571f4a431382ce988a64b"} Jul 02 19:11:32.018007 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:32.017973 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dwn48" event={"ID":"b650207c-2f59-4141-80f1-2cd88e81c5e9","Type":"ContainerStarted","Data":"5ede73543e35a2661fead38f6cb5860428c8cded914ac5099900f806b05bd293"} Jul 02 19:11:32.021514 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:32.021465 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c5wmt" event={"ID":"4a8d6b85-07f6-4d4c-86d4-3dadb68f48af","Type":"ContainerStarted","Data":"ee8bf3ef150fe980021b5ec402f901853e970954ac7bea20104f1e4c8f106cf0"} Jul 02 19:11:32.031656 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:32.031601 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-104.ec2.internal" event={"ID":"d9ea7e94a50077843ac883931f2f632a","Type":"ContainerStarted","Data":"285d5a013d807d24fa32e48a1bd3ba6341dfa1b7218caf75b896b604369a1527"} Jul 02 19:11:32.039954 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:32.039920 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sv2qm" event={"ID":"dee24e3b-8c4a-48d3-9c21-31c9ca499d72","Type":"ContainerStarted","Data":"8f0f1a5c0ec680161cf90476e834bf8b8011de88c4a1a6e0c2aaa94efd6dea67"} Jul 02 19:11:32.052899 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:32.052853 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p8crb" event={"ID":"205e2e72-bfab-43b6-b8d3-8af9a1c8561a","Type":"ContainerStarted","Data":"87e974a638fd1cb898dc10552778c932060c19cb78d0181509ee31154f385ab5"} Jul 02 19:11:33.079384 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:33.079201 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal" event={"ID":"21f1b9fd1a90843c87518bdaf702a316","Type":"ContainerStarted","Data":"424bad5d239302fc49ca166667d6cf79df9c484a8afd3e9fca9b91fa4d86054f"} Jul 02 19:11:33.096918 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:33.095546 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-104.ec2.internal" podStartSLOduration=4.095522876 podStartE2EDuration="4.095522876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:11:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:32.062391065 +0000 UTC m=+4.609116073" watchObservedRunningTime="2026-07-02 19:11:33.095522876 +0000 UTC m=+5.642247865" Jul 02 19:11:33.482783 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:33.481975 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret\") pod \"global-pull-secret-syncer-bhgfb\" (UID: \"882fce17-202b-44a2-9002-efdcb888f969\") " pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:33.482783 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:33.482060 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs\") pod \"network-metrics-daemon-pjsd8\" (UID: \"75464ead-f0fb-4ea8-beba-55b639087860\") " pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:33.482783 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:33.482186 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:11:33.482783 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:33.482248 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs podName:75464ead-f0fb-4ea8-beba-55b639087860 nodeName:}" failed. No retries permitted until 2026-07-02 19:11:37.482229825 +0000 UTC m=+10.028954816 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs") pod "network-metrics-daemon-pjsd8" (UID: "75464ead-f0fb-4ea8-beba-55b639087860") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:11:33.482783 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:33.482640 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:11:33.482783 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:33.482690 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret podName:882fce17-202b-44a2-9002-efdcb888f969 nodeName:}" failed. No retries permitted until 2026-07-02 19:11:37.482675136 +0000 UTC m=+10.029400123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret") pod "global-pull-secret-syncer-bhgfb" (UID: "882fce17-202b-44a2-9002-efdcb888f969") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:11:33.684461 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:33.683809 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2gss\" (UniqueName: \"kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss\") pod \"network-check-target-q6q7s\" (UID: \"c8e6d112-aeda-4479-9158-3e91f551e95c\") " pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:33.684461 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:33.683991 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:11:33.684461 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:33.684009 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:11:33.684461 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:33.684021 2556 projected.go:196] Error preparing data for projected volume kube-api-access-d2gss for pod openshift-network-diagnostics/network-check-target-q6q7s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:11:33.684461 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:33.684079 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss podName:c8e6d112-aeda-4479-9158-3e91f551e95c nodeName:}" failed. No retries permitted until 2026-07-02 19:11:37.684060587 +0000 UTC m=+10.230785593 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-d2gss" (UniqueName: "kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss") pod "network-check-target-q6q7s" (UID: "c8e6d112-aeda-4479-9158-3e91f551e95c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:11:33.972558 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:33.972523 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:33.972558 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:33.972543 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:33.972748 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:33.972524 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:33.972748 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:33.972647 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:11:33.972825 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:33.972752 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhgfb" podUID="882fce17-202b-44a2-9002-efdcb888f969" Jul 02 19:11:33.972856 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:33.972833 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6q7s" podUID="c8e6d112-aeda-4479-9158-3e91f551e95c" Jul 02 19:11:35.972936 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:35.972896 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:35.972936 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:35.972913 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:35.972936 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:35.972942 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:35.973507 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:35.973062 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6q7s" podUID="c8e6d112-aeda-4479-9158-3e91f551e95c" Jul 02 19:11:35.973507 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:35.973205 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhgfb" podUID="882fce17-202b-44a2-9002-efdcb888f969" Jul 02 19:11:35.973507 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:35.973301 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:11:37.518289 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:37.518253 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs\") pod \"network-metrics-daemon-pjsd8\" (UID: \"75464ead-f0fb-4ea8-beba-55b639087860\") " pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:37.518725 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:37.518319 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret\") pod \"global-pull-secret-syncer-bhgfb\" (UID: \"882fce17-202b-44a2-9002-efdcb888f969\") " pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:37.518725 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:37.518379 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:11:37.518725 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:37.518419 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:11:37.518725 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:37.518450 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs podName:75464ead-f0fb-4ea8-beba-55b639087860 nodeName:}" failed. No retries permitted until 2026-07-02 19:11:45.518430421 +0000 UTC m=+18.065155424 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs") pod "network-metrics-daemon-pjsd8" (UID: "75464ead-f0fb-4ea8-beba-55b639087860") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:11:37.518725 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:37.518470 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret podName:882fce17-202b-44a2-9002-efdcb888f969 nodeName:}" failed. No retries permitted until 2026-07-02 19:11:45.518460476 +0000 UTC m=+18.065185464 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret") pod "global-pull-secret-syncer-bhgfb" (UID: "882fce17-202b-44a2-9002-efdcb888f969") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:11:37.719753 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:37.719713 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2gss\" (UniqueName: \"kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss\") pod \"network-check-target-q6q7s\" (UID: \"c8e6d112-aeda-4479-9158-3e91f551e95c\") " pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:37.719963 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:37.719941 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:11:37.720031 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:37.719972 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:11:37.720031 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:37.719987 2556 projected.go:196] Error preparing data for projected volume kube-api-access-d2gss for pod openshift-network-diagnostics/network-check-target-q6q7s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:11:37.720116 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:37.720057 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss podName:c8e6d112-aeda-4479-9158-3e91f551e95c nodeName:}" failed. No retries permitted until 2026-07-02 19:11:45.720036662 +0000 UTC m=+18.266761652 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-d2gss" (UniqueName: "kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss") pod "network-check-target-q6q7s" (UID: "c8e6d112-aeda-4479-9158-3e91f551e95c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:11:37.976298 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:37.976266 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:37.976506 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:37.976367 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:37.976506 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:37.976378 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6q7s" podUID="c8e6d112-aeda-4479-9158-3e91f551e95c" Jul 02 19:11:37.976506 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:37.976403 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:37.976506 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:37.976495 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:11:37.976696 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:37.976562 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhgfb" podUID="882fce17-202b-44a2-9002-efdcb888f969" Jul 02 19:11:39.972722 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:39.972650 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:39.972722 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:39.972668 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:39.973281 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:39.972792 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6q7s" podUID="c8e6d112-aeda-4479-9158-3e91f551e95c" Jul 02 19:11:39.973281 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:39.972812 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:39.973281 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:39.972901 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhgfb" podUID="882fce17-202b-44a2-9002-efdcb888f969" Jul 02 19:11:39.973281 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:39.973000 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:11:41.972535 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:41.972500 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:41.972991 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:41.972635 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:11:41.972991 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:41.972982 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:41.973312 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:41.973115 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhgfb" podUID="882fce17-202b-44a2-9002-efdcb888f969" Jul 02 19:11:41.973312 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:41.973165 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:41.973312 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:41.973271 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6q7s" podUID="c8e6d112-aeda-4479-9158-3e91f551e95c" Jul 02 19:11:42.100175 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:42.100125 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7skzd" event={"ID":"254f17a4-cbdb-420e-9fa7-d2a49ac7c026","Type":"ContainerStarted","Data":"d64ea8c8e87453491e92369295e74dd8f11010c88af0ff8b72be063b603ab3a9"} Jul 02 19:11:42.126038 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:42.125980 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-104.ec2.internal" podStartSLOduration=13.125960575 podStartE2EDuration="13.125960575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:11:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:33.096885048 +0000 UTC m=+5.643610056" watchObservedRunningTime="2026-07-02 19:11:42.125960575 +0000 UTC m=+14.672685583" Jul 02 19:11:43.973043 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:43.973003 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:43.973509 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:43.973003 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:43.973509 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:43.973188 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:11:43.973509 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:43.973219 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhgfb" podUID="882fce17-202b-44a2-9002-efdcb888f969" Jul 02 19:11:43.973509 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:43.973003 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:43.973509 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:43.973319 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6q7s" podUID="c8e6d112-aeda-4479-9158-3e91f551e95c" Jul 02 19:11:45.579264 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:45.579028 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs\") pod \"network-metrics-daemon-pjsd8\" (UID: \"75464ead-f0fb-4ea8-beba-55b639087860\") " pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:45.579264 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:45.579247 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret\") pod \"global-pull-secret-syncer-bhgfb\" (UID: \"882fce17-202b-44a2-9002-efdcb888f969\") " pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:45.579657 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:45.579190 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:11:45.579657 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:45.579334 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:11:45.579657 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:45.579350 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs podName:75464ead-f0fb-4ea8-beba-55b639087860 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:01.579332065 +0000 UTC m=+34.126057051 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs") pod "network-metrics-daemon-pjsd8" (UID: "75464ead-f0fb-4ea8-beba-55b639087860") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:11:45.579657 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:45.579371 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret podName:882fce17-202b-44a2-9002-efdcb888f969 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:01.579360962 +0000 UTC m=+34.126085946 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret") pod "global-pull-secret-syncer-bhgfb" (UID: "882fce17-202b-44a2-9002-efdcb888f969") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:11:45.780543 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:45.780513 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2gss\" (UniqueName: \"kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss\") pod \"network-check-target-q6q7s\" (UID: \"c8e6d112-aeda-4479-9158-3e91f551e95c\") " pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:45.780746 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:45.780700 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:11:45.780746 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:45.780727 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:11:45.780746 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:45.780739 2556 projected.go:196] Error preparing data for projected volume kube-api-access-d2gss for pod openshift-network-diagnostics/network-check-target-q6q7s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:11:45.780912 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:45.780804 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss podName:c8e6d112-aeda-4479-9158-3e91f551e95c nodeName:}" failed. No retries permitted until 2026-07-02 19:12:01.780785378 +0000 UTC m=+34.327510380 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-d2gss" (UniqueName: "kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss") pod "network-check-target-q6q7s" (UID: "c8e6d112-aeda-4479-9158-3e91f551e95c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:11:45.972406 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:45.972369 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:45.972586 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:45.972369 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:45.972586 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:45.972497 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6q7s" podUID="c8e6d112-aeda-4479-9158-3e91f551e95c" Jul 02 19:11:45.972586 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:45.972511 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:45.972746 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:45.972611 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:11:45.972746 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:45.972709 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhgfb" podUID="882fce17-202b-44a2-9002-efdcb888f969" Jul 02 19:11:46.112515 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:46.112467 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sv2qm" event={"ID":"dee24e3b-8c4a-48d3-9c21-31c9ca499d72","Type":"ContainerStarted","Data":"1f957aa871c53c8ecdda66f9201bdae4bc9dce7d83126e2443563753d901e48f"} Jul 02 19:11:46.114218 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:46.114185 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p8crb" event={"ID":"205e2e72-bfab-43b6-b8d3-8af9a1c8561a","Type":"ContainerStarted","Data":"e95ed2660caed4147e2a2c31f01266c09c31a088abe4f046a8869e8413ee6b41"} Jul 02 19:11:46.115672 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:46.115649 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" event={"ID":"92464f4c-5503-4d92-80b5-053bd4f3f041","Type":"ContainerStarted","Data":"afeae2ee58034fe57a5cfb1e56d0833993cab795282efecd78131e0c86836d7b"} Jul 02 19:11:46.117089 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:46.117066 2556 generic.go:356] "Generic (PLEG): container finished" podID="254f17a4-cbdb-420e-9fa7-d2a49ac7c026" containerID="d64ea8c8e87453491e92369295e74dd8f11010c88af0ff8b72be063b603ab3a9" exitCode=0 Jul 02 19:11:46.117188 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:46.117130 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7skzd" event={"ID":"254f17a4-cbdb-420e-9fa7-d2a49ac7c026","Type":"ContainerDied","Data":"d64ea8c8e87453491e92369295e74dd8f11010c88af0ff8b72be063b603ab3a9"} Jul 02 19:11:46.118732 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:46.118719 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dwn48" event={"ID":"b650207c-2f59-4141-80f1-2cd88e81c5e9","Type":"ContainerStarted","Data":"4bfca1b1d800e15dffd168b24382254f60a5f363c33f7cfc88b56731b45096f7"} Jul 02 19:11:46.120344 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:46.120318 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c5wmt" event={"ID":"4a8d6b85-07f6-4d4c-86d4-3dadb68f48af","Type":"ContainerStarted","Data":"ddd0494b1556af4bb2e0ecdbb3b880d05c0f95d73f13b2bfd5ccc23a0b09f04b"} Jul 02 19:11:46.153388 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:46.153331 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dwn48" podStartSLOduration=8.034733558 podStartE2EDuration="18.153311269s" totalImagesPullingTime="10.118577711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:46.15286714 +0000 UTC m=+18.699592176" watchObservedRunningTime="2026-07-02 19:11:46.153311269 +0000 UTC m=+18.700036277" Jul 02 19:11:46.153525 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:46.153437 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sv2qm" podStartSLOduration=8.059761538 podStartE2EDuration="18.153430358s" totalImagesPullingTime="10.09366882s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:46.13365239 +0000 UTC m=+18.680377420" watchObservedRunningTime="2026-07-02 19:11:46.153430358 +0000 UTC m=+18.700155365" Jul 02 19:11:46.171541 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:46.171487 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-c5wmt" podStartSLOduration=8.364310318 podStartE2EDuration="18.171468308s" totalImagesPullingTime="9.80715799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:46.171273076 +0000 UTC m=+18.717998083" watchObservedRunningTime="2026-07-02 19:11:46.171468308 +0000 UTC m=+18.718193316" Jul 02 19:11:46.192776 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:46.192724 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-p8crb" podStartSLOduration=8.096012418 podStartE2EDuration="18.19271085s" totalImagesPullingTime="10.096698432s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:46.192394865 +0000 UTC m=+18.739119874" watchObservedRunningTime="2026-07-02 19:11:46.19271085 +0000 UTC m=+18.739435857" Jul 02 19:11:46.916393 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:46.916312 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-c5wmt" Jul 02 19:11:46.917030 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:46.916930 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-c5wmt" Jul 02 19:11:47.124294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:47.124244 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5m2z2" event={"ID":"4b69d5a7-876e-445e-b851-69b5e139d438","Type":"ContainerStarted","Data":"89b1652c41dfe4f6556fd4432924108084c3a9e73c4872ca4d33cb2f3483e09e"} Jul 02 19:11:47.143013 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:47.142962 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5m2z2" podStartSLOduration=8.050526168 podStartE2EDuration="19.142944931s" totalImagesPullingTime="11.092418763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:47.142782562 +0000 UTC m=+19.689507572" watchObservedRunningTime="2026-07-02 19:11:47.142944931 +0000 UTC m=+19.689669940" Jul 02 19:11:47.973248 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:47.973206 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:47.973248 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:47.973248 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:47.973890 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:47.973326 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:11:47.973890 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:47.973376 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhgfb" podUID="882fce17-202b-44a2-9002-efdcb888f969" Jul 02 19:11:47.973890 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:47.973431 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:47.973890 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:47.973493 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6q7s" podUID="c8e6d112-aeda-4479-9158-3e91f551e95c" Jul 02 19:11:48.126180 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:48.126137 2556 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 02 19:11:49.489365 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:49.489334 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-c5wmt" Jul 02 19:11:49.489992 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:49.489500 2556 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 02 19:11:49.490653 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:49.490633 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-c5wmt" Jul 02 19:11:49.973337 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:49.973192 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:49.973480 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:49.973192 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:49.973480 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:49.973449 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:11:49.973561 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:49.973206 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:49.973561 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:49.973483 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhgfb" podUID="882fce17-202b-44a2-9002-efdcb888f969" Jul 02 19:11:49.973624 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:49.973581 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6q7s" podUID="c8e6d112-aeda-4479-9158-3e91f551e95c" Jul 02 19:11:50.132893 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:50.132857 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fqqsq" event={"ID":"5d0af660-54fe-44aa-b75e-a0116a2f836d","Type":"ContainerStarted","Data":"add4fb78fbe404f77d3b30813e541effa7ae1eb0b3cd0ba03394a7c260c4200e"} Jul 02 19:11:50.135921 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:50.135890 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" event={"ID":"263d1a36-f35e-47e0-b7b4-696f5ebc8621","Type":"ContainerStarted","Data":"05be0f0084dfe1d3be0150a6dad8ddaa74aa39854a04faa5c346571d2b321eae"} Jul 02 19:11:50.136047 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:50.135977 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" event={"ID":"263d1a36-f35e-47e0-b7b4-696f5ebc8621","Type":"ContainerStarted","Data":"86dcdb1ac9a3988a6321c9126218c9372f77a3210bfa50f0a85ded2b0a0dcffe"} Jul 02 19:11:50.848923 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:50.848713 2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 19:11:50.919390 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:50.919302 2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T19:11:50.848918506Z","UUID":"51f1f357-0543-472a-bf84-3f4f6bb5a62d","Handler":null,"Name":"","Endpoint":""} Jul 02 19:11:50.921519 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:50.921498 2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 19:11:50.921624 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:50.921533 2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 19:11:51.140547 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:51.140464 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" event={"ID":"263d1a36-f35e-47e0-b7b4-696f5ebc8621","Type":"ContainerStarted","Data":"e22f3e71a3beec99c68d06b3f7a1010341da1afa3cfddcaa3792bfefa908cf99"} Jul 02 19:11:51.140702 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:51.140569 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" event={"ID":"263d1a36-f35e-47e0-b7b4-696f5ebc8621","Type":"ContainerStarted","Data":"ae1f3a9ec5fd56428766bac2e58ef23122960256cbbbdd88ec52beec05a39764"} Jul 02 19:11:51.140702 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:51.140611 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" event={"ID":"263d1a36-f35e-47e0-b7b4-696f5ebc8621","Type":"ContainerStarted","Data":"622ceaf9097925ed6f25e183bd48413b74a6b032a577d50e4ab6288e1c1cb1a1"} Jul 02 19:11:51.140702 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:51.140654 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" event={"ID":"263d1a36-f35e-47e0-b7b4-696f5ebc8621","Type":"ContainerStarted","Data":"6111af0987f35ad058781a815ec1827f0191d971de083f6716501b526bddaa40"} Jul 02 19:11:51.142072 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:51.142047 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" event={"ID":"92464f4c-5503-4d92-80b5-053bd4f3f041","Type":"ContainerStarted","Data":"d200e57ed237d266fcc19611e352b7b20901ce213f0d915a6e331942ce258093"} Jul 02 19:11:51.973203 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:51.973169 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:51.973822 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:51.973177 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:51.973822 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:51.973289 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:11:51.973822 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:51.973186 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:51.973822 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:51.973351 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhgfb" podUID="882fce17-202b-44a2-9002-efdcb888f969" Jul 02 19:11:51.973822 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:51.973430 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6q7s" podUID="c8e6d112-aeda-4479-9158-3e91f551e95c" Jul 02 19:11:53.148605 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:53.148569 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" event={"ID":"263d1a36-f35e-47e0-b7b4-696f5ebc8621","Type":"ContainerStarted","Data":"836b787eb82964861839ef3eec019975614be34257e31c870cf91c52f3a17a0d"} Jul 02 19:11:53.150428 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:53.150405 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" event={"ID":"92464f4c-5503-4d92-80b5-053bd4f3f041","Type":"ContainerStarted","Data":"733076bc9062916339c7d4a25429479c2ac9f42442a6d75e58e38fe5a8ae5220"} Jul 02 19:11:53.169092 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:53.169050 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fqqsq" podStartSLOduration=5.396321482 podStartE2EDuration="25.169034669s" totalImagesPullingTime="19.772713187s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:50.152472624 +0000 UTC m=+22.699197634" watchObservedRunningTime="2026-07-02 19:11:53.169034669 +0000 UTC m=+25.715759745" Jul 02 19:11:53.169242 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:53.169222 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92zcl" podStartSLOduration=8.884224996 podStartE2EDuration="25.169217456s" totalImagesPullingTime="16.28499246s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:11:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:53.168458424 +0000 UTC m=+25.715183442" watchObservedRunningTime="2026-07-02 19:11:53.169217456 +0000 UTC m=+25.715942463" Jul 02 19:11:53.972229 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:53.972200 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:53.972229 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:53.972218 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:53.972469 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:53.972199 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:53.972469 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:53.972329 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhgfb" podUID="882fce17-202b-44a2-9002-efdcb888f969" Jul 02 19:11:53.972469 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:53.972411 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:11:53.972649 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:53.972496 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6q7s" podUID="c8e6d112-aeda-4479-9158-3e91f551e95c" Jul 02 19:11:55.158826 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:55.158554 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" event={"ID":"263d1a36-f35e-47e0-b7b4-696f5ebc8621","Type":"ContainerStarted","Data":"6639c56dcd55ade2dfc10cd186884326ea8b74277cc46ee1d4dddc3b0770dcec"} Jul 02 19:11:55.159437 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:55.158898 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:55.159437 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:55.158968 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:55.159437 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:55.159015 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:55.175490 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:55.175464 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:55.175826 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:55.175811 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:11:55.189446 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:55.189398 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" podStartSLOduration=8.371951398 podStartE2EDuration="27.189383752s" totalImagesPullingTime="18.817432354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:55.18907411 +0000 UTC m=+27.735799117" watchObservedRunningTime="2026-07-02 19:11:55.189383752 +0000 UTC m=+27.736108759" Jul 02 19:11:55.973057 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:55.973026 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:55.973269 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:55.973071 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:55.973269 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:55.973040 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:55.973269 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:55.973179 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:11:55.973417 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:55.973268 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhgfb" podUID="882fce17-202b-44a2-9002-efdcb888f969" Jul 02 19:11:55.973417 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:55.973383 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6q7s" podUID="c8e6d112-aeda-4479-9158-3e91f551e95c" Jul 02 19:11:57.226490 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:57.226458 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q6q7s"] Jul 02 19:11:57.226912 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:57.226631 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:57.226912 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:57.226723 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6q7s" podUID="c8e6d112-aeda-4479-9158-3e91f551e95c" Jul 02 19:11:57.229814 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:57.229694 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bhgfb"] Jul 02 19:11:57.229814 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:57.229805 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:57.229995 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:57.229897 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhgfb" podUID="882fce17-202b-44a2-9002-efdcb888f969" Jul 02 19:11:57.230473 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:57.230449 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pjsd8"] Jul 02 19:11:57.230575 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:57.230553 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:57.230676 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:57.230658 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:11:58.166197 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:58.166164 2556 generic.go:356] "Generic (PLEG): container finished" podID="254f17a4-cbdb-420e-9fa7-d2a49ac7c026" containerID="97cf6e82a50c4335b52181cd6fa8a1335fc7647cfb8effb5bac419fa224395a0" exitCode=0 Jul 02 19:11:58.166357 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:58.166216 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7skzd" event={"ID":"254f17a4-cbdb-420e-9fa7-d2a49ac7c026","Type":"ContainerDied","Data":"97cf6e82a50c4335b52181cd6fa8a1335fc7647cfb8effb5bac419fa224395a0"} Jul 02 19:11:58.972705 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:58.972673 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:11:58.973081 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:58.972673 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:11:58.973081 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:58.972789 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:11:58.973081 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:58.972689 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:11:58.973081 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:58.972863 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhgfb" podUID="882fce17-202b-44a2-9002-efdcb888f969" Jul 02 19:11:58.973081 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:11:58.972918 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6q7s" podUID="c8e6d112-aeda-4479-9158-3e91f551e95c" Jul 02 19:11:59.170177 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:59.170126 2556 generic.go:356] "Generic (PLEG): container finished" podID="254f17a4-cbdb-420e-9fa7-d2a49ac7c026" containerID="4045a7405606d7ec20210e4cef106ede1407a7891e8152ba79144fc2bfbfcf6f" exitCode=0 Jul 02 19:11:59.170313 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:11:59.170190 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7skzd" event={"ID":"254f17a4-cbdb-420e-9fa7-d2a49ac7c026","Type":"ContainerDied","Data":"4045a7405606d7ec20210e4cef106ede1407a7891e8152ba79144fc2bfbfcf6f"} Jul 02 19:12:00.174368 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:00.174177 2556 generic.go:356] "Generic (PLEG): container finished" podID="254f17a4-cbdb-420e-9fa7-d2a49ac7c026" containerID="564d358470a3f48ec91e13d5c85bc2c07553d672059ab9634189d1334ae2139f" exitCode=0 Jul 02 19:12:00.174993 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:00.174249 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7skzd" event={"ID":"254f17a4-cbdb-420e-9fa7-d2a49ac7c026","Type":"ContainerDied","Data":"564d358470a3f48ec91e13d5c85bc2c07553d672059ab9634189d1334ae2139f"} Jul 02 19:12:00.972190 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:00.972162 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:12:00.972420 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:00.972268 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:12:00.972420 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:00.972293 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:12:00.972420 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:00.972283 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6q7s" podUID="c8e6d112-aeda-4479-9158-3e91f551e95c" Jul 02 19:12:00.972420 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:00.972392 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:12:00.972543 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:00.972467 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhgfb" podUID="882fce17-202b-44a2-9002-efdcb888f969" Jul 02 19:12:01.594517 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:01.594477 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs\") pod \"network-metrics-daemon-pjsd8\" (UID: \"75464ead-f0fb-4ea8-beba-55b639087860\") " pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:12:01.594517 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:01.594524 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret\") pod \"global-pull-secret-syncer-bhgfb\" (UID: \"882fce17-202b-44a2-9002-efdcb888f969\") " pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:12:01.595008 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:01.594608 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:12:01.595008 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:01.594611 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:12:01.595008 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:01.594658 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret podName:882fce17-202b-44a2-9002-efdcb888f969 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:33.594645464 +0000 UTC m=+66.141370450 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret") pod "global-pull-secret-syncer-bhgfb" (UID: "882fce17-202b-44a2-9002-efdcb888f969") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:12:01.595008 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:01.594673 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs podName:75464ead-f0fb-4ea8-beba-55b639087860 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:33.594666177 +0000 UTC m=+66.141391162 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs") pod "network-metrics-daemon-pjsd8" (UID: "75464ead-f0fb-4ea8-beba-55b639087860") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:12:01.796414 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:01.796300 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2gss\" (UniqueName: \"kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss\") pod \"network-check-target-q6q7s\" (UID: \"c8e6d112-aeda-4479-9158-3e91f551e95c\") " pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:12:01.796565 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:01.796468 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:12:01.796565 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:01.796492 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:12:01.796565 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:01.796503 2556 projected.go:196] Error preparing data for projected volume kube-api-access-d2gss for pod openshift-network-diagnostics/network-check-target-q6q7s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:12:01.796565 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:01.796558 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss podName:c8e6d112-aeda-4479-9158-3e91f551e95c nodeName:}" failed. No retries permitted until 2026-07-02 19:12:33.796541248 +0000 UTC m=+66.343266249 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-d2gss" (UniqueName: "kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss") pod "network-check-target-q6q7s" (UID: "c8e6d112-aeda-4479-9158-3e91f551e95c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:12:02.972824 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:02.972784 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:12:02.973294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:02.972792 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:12:02.973294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:02.972940 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:12:02.973294 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:02.972901 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhgfb" podUID="882fce17-202b-44a2-9002-efdcb888f969" Jul 02 19:12:02.973294 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:02.973061 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:12:02.973294 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:02.973142 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q6q7s" podUID="c8e6d112-aeda-4479-9158-3e91f551e95c" Jul 02 19:12:03.714988 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.714957 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-104.ec2.internal" event="NodeReady" Jul 02 19:12:03.715187 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.715089 2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 19:12:03.749830 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.749799 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6cddd996cc-6qvxb"] Jul 02 19:12:03.770922 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.770889 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cddd996cc-6qvxb"] Jul 02 19:12:03.771121 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.771009 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:03.773713 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.773682 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7qq96"] Jul 02 19:12:03.774604 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.774576 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7dhp8\"" Jul 02 19:12:03.774732 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.774619 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 02 19:12:03.775695 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.775668 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 02 19:12:03.777448 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.777428 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 02 19:12:03.781202 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.781181 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 02 19:12:03.794990 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.794950 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8t8d8"] Jul 02 19:12:03.795250 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.795107 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7qq96" Jul 02 19:12:03.799171 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.799059 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 19:12:03.799303 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.799223 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 19:12:03.800543 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.800487 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9snh5\"" Jul 02 19:12:03.814500 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.814466 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl"] Jul 02 19:12:03.814631 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.814515 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8t8d8" Jul 02 19:12:03.817477 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.817455 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gv77g\"" Jul 02 19:12:03.818392 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.817982 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 19:12:03.818392 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.818007 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 19:12:03.819015 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.818543 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 19:12:03.842674 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.842642 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8t8d8"] Jul 02 19:12:03.842910 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.842886 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7qq96"] Jul 02 19:12:03.843015 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.842753 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:12:03.843015 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.842926 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl"] Jul 02 19:12:03.845565 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.845546 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 02 19:12:03.845781 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.845578 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 02 19:12:03.845781 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.845592 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-86ffp\"" Jul 02 19:12:03.910915 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.910879 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/768126ea-6b59-47d9-a137-b71cd0695d33-registry-certificates\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:03.911083 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.910925 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:03.911083 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.910965 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/768126ea-6b59-47d9-a137-b71cd0695d33-trusted-ca\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:03.911083 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.911054 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/768126ea-6b59-47d9-a137-b71cd0695d33-installation-pull-secrets\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:03.911237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.911112 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/768126ea-6b59-47d9-a137-b71cd0695d33-ca-trust-extracted\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:03.911237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.911159 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s5ws\" (UniqueName: \"kubernetes.io/projected/77f8b592-4af7-48fe-9385-0de92afaf07c-kube-api-access-6s5ws\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:12:03.911237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.911188 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-bound-sa-token\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:03.911237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.911211 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66zt7\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-kube-api-access-66zt7\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:03.911428 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.911264 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77f8b592-4af7-48fe-9385-0de92afaf07c-tmp-dir\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:12:03.911428 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.911287 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/768126ea-6b59-47d9-a137-b71cd0695d33-image-registry-private-configuration\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:03.911428 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.911309 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:12:03.911428 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.911403 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/77f8b592-4af7-48fe-9385-0de92afaf07c-config-volume\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:12:03.949962 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:03.949931 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5wmt_4a8d6b85-07f6-4d4c-86d4-3dadb68f48af/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:04.011930 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.011844 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/768126ea-6b59-47d9-a137-b71cd0695d33-installation-pull-secrets\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:04.011930 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.011889 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/768126ea-6b59-47d9-a137-b71cd0695d33-ca-trust-extracted\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:04.011930 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.011910 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-x6qcl\" (UID: \"535b962d-6d94-4e02-bd83-4411a9f5fd05\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:12:04.012488 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.011958 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s5ws\" (UniqueName: \"kubernetes.io/projected/77f8b592-4af7-48fe-9385-0de92afaf07c-kube-api-access-6s5ws\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:12:04.012488 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.012031 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-bound-sa-token\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:04.012488 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.012058 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66zt7\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-kube-api-access-66zt7\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:04.012488 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.012094 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77f8b592-4af7-48fe-9385-0de92afaf07c-tmp-dir\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:12:04.012488 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.012122 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/768126ea-6b59-47d9-a137-b71cd0695d33-image-registry-private-configuration\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:04.012488 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.012170 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/171c244d-2dda-414a-a89f-c56e1a47c97c-cert\") pod \"ingress-canary-8t8d8\" (UID: \"171c244d-2dda-414a-a89f-c56e1a47c97c\") " pod="openshift-ingress-canary/ingress-canary-8t8d8" Jul 02 19:12:04.012488 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.012192 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h69vt\" (UniqueName: \"kubernetes.io/projected/171c244d-2dda-414a-a89f-c56e1a47c97c-kube-api-access-h69vt\") pod \"ingress-canary-8t8d8\" (UID: \"171c244d-2dda-414a-a89f-c56e1a47c97c\") " pod="openshift-ingress-canary/ingress-canary-8t8d8" Jul 02 19:12:04.012488 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.012219 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:12:04.012488 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.012251 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/77f8b592-4af7-48fe-9385-0de92afaf07c-config-volume\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:12:04.012488 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.012273 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/768126ea-6b59-47d9-a137-b71cd0695d33-registry-certificates\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:04.012488 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.012297 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/768126ea-6b59-47d9-a137-b71cd0695d33-ca-trust-extracted\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:04.012488 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.012302 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:04.012488 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.012362 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/768126ea-6b59-47d9-a137-b71cd0695d33-trusted-ca\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:04.012488 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:04.012373 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 19:12:04.012488 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:04.012388 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cddd996cc-6qvxb: secret "image-registry-tls" not found Jul 02 19:12:04.013122 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.012393 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/535b962d-6d94-4e02-bd83-4411a9f5fd05-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-x6qcl\" (UID: \"535b962d-6d94-4e02-bd83-4411a9f5fd05\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:12:04.013122 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:04.012447 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls podName:768126ea-6b59-47d9-a137-b71cd0695d33 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:04.512430059 +0000 UTC m=+37.059155048 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls") pod "image-registry-6cddd996cc-6qvxb" (UID: "768126ea-6b59-47d9-a137-b71cd0695d33") : secret "image-registry-tls" not found Jul 02 19:12:04.013122 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.012451 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77f8b592-4af7-48fe-9385-0de92afaf07c-tmp-dir\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:12:04.013122 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:04.012575 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:12:04.013122 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:04.012646 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls podName:77f8b592-4af7-48fe-9385-0de92afaf07c nodeName:}" failed. No retries permitted until 2026-07-02 19:12:04.512627749 +0000 UTC m=+37.059352748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls") pod "dns-default-7qq96" (UID: "77f8b592-4af7-48fe-9385-0de92afaf07c") : secret "dns-default-metrics-tls" not found Jul 02 19:12:04.013122 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.012784 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/77f8b592-4af7-48fe-9385-0de92afaf07c-config-volume\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:12:04.013122 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.012945 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/768126ea-6b59-47d9-a137-b71cd0695d33-registry-certificates\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:04.013429 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.013400 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/768126ea-6b59-47d9-a137-b71cd0695d33-trusted-ca\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:04.016477 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.016451 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/768126ea-6b59-47d9-a137-b71cd0695d33-image-registry-private-configuration\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:04.016625 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.016459 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/768126ea-6b59-47d9-a137-b71cd0695d33-installation-pull-secrets\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:04.026896 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.026874 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66zt7\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-kube-api-access-66zt7\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:04.027087 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.027070 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s5ws\" (UniqueName: \"kubernetes.io/projected/77f8b592-4af7-48fe-9385-0de92afaf07c-kube-api-access-6s5ws\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:12:04.027186 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.027140 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-bound-sa-token\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:04.062369 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.062331 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5wmt_4a8d6b85-07f6-4d4c-86d4-3dadb68f48af/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:04.113611 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.113569 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/171c244d-2dda-414a-a89f-c56e1a47c97c-cert\") pod \"ingress-canary-8t8d8\" (UID: \"171c244d-2dda-414a-a89f-c56e1a47c97c\") " pod="openshift-ingress-canary/ingress-canary-8t8d8" Jul 02 19:12:04.113611 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.113610 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h69vt\" (UniqueName: \"kubernetes.io/projected/171c244d-2dda-414a-a89f-c56e1a47c97c-kube-api-access-h69vt\") pod \"ingress-canary-8t8d8\" (UID: \"171c244d-2dda-414a-a89f-c56e1a47c97c\") " pod="openshift-ingress-canary/ingress-canary-8t8d8" Jul 02 19:12:04.113830 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.113676 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/535b962d-6d94-4e02-bd83-4411a9f5fd05-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-x6qcl\" (UID: \"535b962d-6d94-4e02-bd83-4411a9f5fd05\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:12:04.113830 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.113734 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-x6qcl\" (UID: \"535b962d-6d94-4e02-bd83-4411a9f5fd05\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:12:04.113830 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:04.113811 2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 19:12:04.113983 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:04.113892 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert podName:535b962d-6d94-4e02-bd83-4411a9f5fd05 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:04.613874433 +0000 UTC m=+37.160599420 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-x6qcl" (UID: "535b962d-6d94-4e02-bd83-4411a9f5fd05") : secret "networking-console-plugin-cert" not found Jul 02 19:12:04.114446 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.114423 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/535b962d-6d94-4e02-bd83-4411a9f5fd05-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-x6qcl\" (UID: \"535b962d-6d94-4e02-bd83-4411a9f5fd05\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:12:04.116356 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.116333 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/171c244d-2dda-414a-a89f-c56e1a47c97c-cert\") pod \"ingress-canary-8t8d8\" (UID: \"171c244d-2dda-414a-a89f-c56e1a47c97c\") " pod="openshift-ingress-canary/ingress-canary-8t8d8" Jul 02 19:12:04.127249 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.127217 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h69vt\" (UniqueName: \"kubernetes.io/projected/171c244d-2dda-414a-a89f-c56e1a47c97c-kube-api-access-h69vt\") pod \"ingress-canary-8t8d8\" (UID: \"171c244d-2dda-414a-a89f-c56e1a47c97c\") " pod="openshift-ingress-canary/ingress-canary-8t8d8" Jul 02 19:12:04.425403 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.425365 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8t8d8" Jul 02 19:12:04.516193 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.516139 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:04.516394 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.516310 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:12:04.516394 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:04.516312 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 19:12:04.516394 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:04.516335 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cddd996cc-6qvxb: secret "image-registry-tls" not found Jul 02 19:12:04.516394 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:04.516396 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls podName:768126ea-6b59-47d9-a137-b71cd0695d33 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:05.51637782 +0000 UTC m=+38.063102804 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls") pod "image-registry-6cddd996cc-6qvxb" (UID: "768126ea-6b59-47d9-a137-b71cd0695d33") : secret "image-registry-tls" not found Jul 02 19:12:04.516605 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:04.516410 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:12:04.516605 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:04.516451 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls podName:77f8b592-4af7-48fe-9385-0de92afaf07c nodeName:}" failed. No retries permitted until 2026-07-02 19:12:05.51643822 +0000 UTC m=+38.063163205 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls") pod "dns-default-7qq96" (UID: "77f8b592-4af7-48fe-9385-0de92afaf07c") : secret "dns-default-metrics-tls" not found Jul 02 19:12:04.617019 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.616985 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-x6qcl\" (UID: \"535b962d-6d94-4e02-bd83-4411a9f5fd05\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:12:04.617218 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:04.617190 2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 19:12:04.617288 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:04.617274 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert podName:535b962d-6d94-4e02-bd83-4411a9f5fd05 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:05.617252231 +0000 UTC m=+38.163977222 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-x6qcl" (UID: "535b962d-6d94-4e02-bd83-4411a9f5fd05") : secret "networking-console-plugin-cert" not found Jul 02 19:12:04.972371 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.972336 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:12:04.972560 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.972336 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:12:04.972674 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.972338 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:12:04.975477 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.975405 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-shdtl\"" Jul 02 19:12:04.975477 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.975429 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 19:12:04.975652 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.975491 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 19:12:04.975652 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.975498 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 19:12:04.975652 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.975631 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 19:12:04.977043 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:04.977014 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bjk9g\"" Jul 02 19:12:05.524597 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:05.524562 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:05.525022 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:05.524652 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:12:05.525022 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:05.524737 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:12:05.525022 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:05.524740 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 19:12:05.525022 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:05.524763 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cddd996cc-6qvxb: secret "image-registry-tls" not found Jul 02 19:12:05.525022 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:05.524787 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls podName:77f8b592-4af7-48fe-9385-0de92afaf07c nodeName:}" failed. No retries permitted until 2026-07-02 19:12:07.524774411 +0000 UTC m=+40.071499397 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls") pod "dns-default-7qq96" (UID: "77f8b592-4af7-48fe-9385-0de92afaf07c") : secret "dns-default-metrics-tls" not found Jul 02 19:12:05.525022 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:05.524821 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls podName:768126ea-6b59-47d9-a137-b71cd0695d33 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:07.524801385 +0000 UTC m=+40.071526390 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls") pod "image-registry-6cddd996cc-6qvxb" (UID: "768126ea-6b59-47d9-a137-b71cd0695d33") : secret "image-registry-tls" not found Jul 02 19:12:05.625473 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:05.625443 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-x6qcl\" (UID: \"535b962d-6d94-4e02-bd83-4411a9f5fd05\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:12:05.625635 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:05.625590 2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 19:12:05.625680 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:05.625652 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert podName:535b962d-6d94-4e02-bd83-4411a9f5fd05 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:07.625637464 +0000 UTC m=+40.172362449 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-x6qcl" (UID: "535b962d-6d94-4e02-bd83-4411a9f5fd05") : secret "networking-console-plugin-cert" not found Jul 02 19:12:05.721388 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:05.721360 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8t8d8"] Jul 02 19:12:06.189295 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:06.189261 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8t8d8" event={"ID":"171c244d-2dda-414a-a89f-c56e1a47c97c","Type":"ContainerStarted","Data":"18c27efe16544598588edaed4b99fdbf3773ef28dd24ab6bdd3ccc7f76c93680"} Jul 02 19:12:06.192010 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:06.191985 2556 generic.go:356] "Generic (PLEG): container finished" podID="254f17a4-cbdb-420e-9fa7-d2a49ac7c026" containerID="46adf3f7a14202818f74fd3c2c4c25ef892f93d3b748da7697d256c6cb6ba54c" exitCode=0 Jul 02 19:12:06.192138 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:06.192020 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7skzd" event={"ID":"254f17a4-cbdb-420e-9fa7-d2a49ac7c026","Type":"ContainerDied","Data":"46adf3f7a14202818f74fd3c2c4c25ef892f93d3b748da7697d256c6cb6ba54c"} Jul 02 19:12:07.197060 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:07.197024 2556 generic.go:356] "Generic (PLEG): container finished" podID="254f17a4-cbdb-420e-9fa7-d2a49ac7c026" containerID="2884646b255d6666a668bc1ecc718509c6cae5805d65adf5359f213bb8f70f86" exitCode=0 Jul 02 19:12:07.197529 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:07.197073 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7skzd" event={"ID":"254f17a4-cbdb-420e-9fa7-d2a49ac7c026","Type":"ContainerDied","Data":"2884646b255d6666a668bc1ecc718509c6cae5805d65adf5359f213bb8f70f86"} Jul 02 19:12:07.539737 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:07.539642 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:12:07.539737 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:07.539707 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:07.539952 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:07.539825 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:12:07.539952 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:07.539876 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 19:12:07.539952 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:07.539891 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cddd996cc-6qvxb: secret "image-registry-tls" not found Jul 02 19:12:07.539952 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:07.539907 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls podName:77f8b592-4af7-48fe-9385-0de92afaf07c nodeName:}" failed. No retries permitted until 2026-07-02 19:12:11.539886456 +0000 UTC m=+44.086611449 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls") pod "dns-default-7qq96" (UID: "77f8b592-4af7-48fe-9385-0de92afaf07c") : secret "dns-default-metrics-tls" not found Jul 02 19:12:07.539952 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:07.539929 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls podName:768126ea-6b59-47d9-a137-b71cd0695d33 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:11.539918467 +0000 UTC m=+44.086643452 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls") pod "image-registry-6cddd996cc-6qvxb" (UID: "768126ea-6b59-47d9-a137-b71cd0695d33") : secret "image-registry-tls" not found Jul 02 19:12:07.640481 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:07.640445 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-x6qcl\" (UID: \"535b962d-6d94-4e02-bd83-4411a9f5fd05\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:12:07.640644 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:07.640583 2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 19:12:07.640683 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:07.640644 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert podName:535b962d-6d94-4e02-bd83-4411a9f5fd05 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:11.640627539 +0000 UTC m=+44.187352544 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-x6qcl" (UID: "535b962d-6d94-4e02-bd83-4411a9f5fd05") : secret "networking-console-plugin-cert" not found Jul 02 19:12:08.204996 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:08.204963 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7skzd" event={"ID":"254f17a4-cbdb-420e-9fa7-d2a49ac7c026","Type":"ContainerStarted","Data":"d54483428aad8f21e729b60c5afd1aae773fca5d11377092c77517527b0799e1"} Jul 02 19:12:08.232715 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:08.232667 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7skzd" podStartSLOduration=11.604719082999999 podStartE2EDuration="40.232651512s" totalImagesPullingTime="28.627932429s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:11:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:08.231116043 +0000 UTC m=+40.777841068" watchObservedRunningTime="2026-07-02 19:12:08.232651512 +0000 UTC m=+40.779376518" Jul 02 19:12:10.210792 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:10.210758 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8t8d8" event={"ID":"171c244d-2dda-414a-a89f-c56e1a47c97c","Type":"ContainerStarted","Data":"e300b50f8ae3fe13e5c7be559301501f9d1bb41b0cdb1cdd330d60c21e553e43"} Jul 02 19:12:10.228108 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:10.228060 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8t8d8" podStartSLOduration=3.6620766099999997 podStartE2EDuration="7.228041865s" totalImagesPullingTime="3.565965255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:10.227633223 +0000 UTC m=+42.774358231" watchObservedRunningTime="2026-07-02 19:12:10.228041865 +0000 UTC m=+42.774766872" Jul 02 19:12:11.568682 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:11.568645 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:12:11.569088 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:11.568691 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:11.569088 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:11.568793 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:12:11.569088 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:11.568815 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 19:12:11.569088 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:11.568825 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cddd996cc-6qvxb: secret "image-registry-tls" not found Jul 02 19:12:11.569088 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:11.568854 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls podName:77f8b592-4af7-48fe-9385-0de92afaf07c nodeName:}" failed. No retries permitted until 2026-07-02 19:12:19.568839641 +0000 UTC m=+52.115564627 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls") pod "dns-default-7qq96" (UID: "77f8b592-4af7-48fe-9385-0de92afaf07c") : secret "dns-default-metrics-tls" not found Jul 02 19:12:11.569088 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:11.568870 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls podName:768126ea-6b59-47d9-a137-b71cd0695d33 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:19.568863329 +0000 UTC m=+52.115588315 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls") pod "image-registry-6cddd996cc-6qvxb" (UID: "768126ea-6b59-47d9-a137-b71cd0695d33") : secret "image-registry-tls" not found Jul 02 19:12:11.669555 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:11.669514 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-x6qcl\" (UID: \"535b962d-6d94-4e02-bd83-4411a9f5fd05\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:12:11.669721 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:11.669654 2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 19:12:11.669721 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:11.669717 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert podName:535b962d-6d94-4e02-bd83-4411a9f5fd05 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:19.669701612 +0000 UTC m=+52.216426596 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-x6qcl" (UID: "535b962d-6d94-4e02-bd83-4411a9f5fd05") : secret "networking-console-plugin-cert" not found Jul 02 19:12:19.626260 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:19.626223 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:12:19.626260 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:19.626265 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:19.626780 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:19.626369 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 19:12:19.626780 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:19.626382 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cddd996cc-6qvxb: secret "image-registry-tls" not found Jul 02 19:12:19.626780 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:19.626425 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls podName:768126ea-6b59-47d9-a137-b71cd0695d33 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:35.626412832 +0000 UTC m=+68.173137816 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls") pod "image-registry-6cddd996cc-6qvxb" (UID: "768126ea-6b59-47d9-a137-b71cd0695d33") : secret "image-registry-tls" not found Jul 02 19:12:19.626780 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:19.626368 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:12:19.626780 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:19.626490 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls podName:77f8b592-4af7-48fe-9385-0de92afaf07c nodeName:}" failed. No retries permitted until 2026-07-02 19:12:35.626476469 +0000 UTC m=+68.173201454 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls") pod "dns-default-7qq96" (UID: "77f8b592-4af7-48fe-9385-0de92afaf07c") : secret "dns-default-metrics-tls" not found Jul 02 19:12:19.726909 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:19.726877 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-x6qcl\" (UID: \"535b962d-6d94-4e02-bd83-4411a9f5fd05\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:12:19.727054 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:19.727024 2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 19:12:19.727101 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:19.727090 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert podName:535b962d-6d94-4e02-bd83-4411a9f5fd05 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:35.727075847 +0000 UTC m=+68.273800831 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-x6qcl" (UID: "535b962d-6d94-4e02-bd83-4411a9f5fd05") : secret "networking-console-plugin-cert" not found Jul 02 19:12:27.172618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:27.172580 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p2j8h" Jul 02 19:12:33.625291 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:33.625254 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs\") pod \"network-metrics-daemon-pjsd8\" (UID: \"75464ead-f0fb-4ea8-beba-55b639087860\") " pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:12:33.625747 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:33.625317 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret\") pod \"global-pull-secret-syncer-bhgfb\" (UID: \"882fce17-202b-44a2-9002-efdcb888f969\") " pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:12:33.628122 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:33.628097 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 19:12:33.628244 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:33.628223 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 19:12:33.636396 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:33.636378 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 02 19:12:33.636461 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:33.636450 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs podName:75464ead-f0fb-4ea8-beba-55b639087860 nodeName:}" failed. No retries permitted until 2026-07-02 19:13:37.636435061 +0000 UTC m=+130.183160048 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs") pod "network-metrics-daemon-pjsd8" (UID: "75464ead-f0fb-4ea8-beba-55b639087860") : secret "metrics-daemon-secret" not found Jul 02 19:12:33.638454 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:33.638438 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/882fce17-202b-44a2-9002-efdcb888f969-original-pull-secret\") pod \"global-pull-secret-syncer-bhgfb\" (UID: \"882fce17-202b-44a2-9002-efdcb888f969\") " pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:12:33.798528 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:33.798483 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhgfb" Jul 02 19:12:33.826957 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:33.826918 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2gss\" (UniqueName: \"kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss\") pod \"network-check-target-q6q7s\" (UID: \"c8e6d112-aeda-4479-9158-3e91f551e95c\") " pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:12:33.829851 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:33.829833 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 19:12:33.840559 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:33.840532 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 19:12:33.869217 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:33.869178 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2gss\" (UniqueName: \"kubernetes.io/projected/c8e6d112-aeda-4479-9158-3e91f551e95c-kube-api-access-d2gss\") pod \"network-check-target-q6q7s\" (UID: \"c8e6d112-aeda-4479-9158-3e91f551e95c\") " pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:12:33.967425 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:33.967385 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bhgfb"] Jul 02 19:12:33.983477 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:12:33.983452 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod882fce17_202b_44a2_9002_efdcb888f969.slice/crio-cc2165d832aefcf437029d60df4b93f49f378b171f784cc9a3cc00e38931ce76 WatchSource:0}: Error finding container cc2165d832aefcf437029d60df4b93f49f378b171f784cc9a3cc00e38931ce76: Status 404 returned error can't find the container with id cc2165d832aefcf437029d60df4b93f49f378b171f784cc9a3cc00e38931ce76 Jul 02 19:12:34.094624 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:34.094586 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bjk9g\"" Jul 02 19:12:34.102753 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:34.102729 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:12:34.222296 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:34.222266 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q6q7s"] Jul 02 19:12:34.225394 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:12:34.225370 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8e6d112_aeda_4479_9158_3e91f551e95c.slice/crio-c85a5515b008ef0dfdaa2892838502bcd4e2ce133a4b72927b4ca97be3bf4a83 WatchSource:0}: Error finding container c85a5515b008ef0dfdaa2892838502bcd4e2ce133a4b72927b4ca97be3bf4a83: Status 404 returned error can't find the container with id c85a5515b008ef0dfdaa2892838502bcd4e2ce133a4b72927b4ca97be3bf4a83 Jul 02 19:12:34.263269 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:34.263240 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bhgfb" event={"ID":"882fce17-202b-44a2-9002-efdcb888f969","Type":"ContainerStarted","Data":"cc2165d832aefcf437029d60df4b93f49f378b171f784cc9a3cc00e38931ce76"} Jul 02 19:12:34.264348 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:34.264326 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q6q7s" event={"ID":"c8e6d112-aeda-4479-9158-3e91f551e95c","Type":"ContainerStarted","Data":"c85a5515b008ef0dfdaa2892838502bcd4e2ce133a4b72927b4ca97be3bf4a83"} Jul 02 19:12:35.641187 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:35.641129 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:12:35.641802 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:35.641205 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:12:35.641802 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:35.641333 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 19:12:35.641802 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:35.641352 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cddd996cc-6qvxb: secret "image-registry-tls" not found Jul 02 19:12:35.641802 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:35.641408 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls podName:768126ea-6b59-47d9-a137-b71cd0695d33 nodeName:}" failed. No retries permitted until 2026-07-02 19:13:07.641390269 +0000 UTC m=+100.188115257 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls") pod "image-registry-6cddd996cc-6qvxb" (UID: "768126ea-6b59-47d9-a137-b71cd0695d33") : secret "image-registry-tls" not found Jul 02 19:12:35.641802 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:35.641331 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:12:35.641802 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:35.641514 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls podName:77f8b592-4af7-48fe-9385-0de92afaf07c nodeName:}" failed. No retries permitted until 2026-07-02 19:13:07.641495345 +0000 UTC m=+100.188220333 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls") pod "dns-default-7qq96" (UID: "77f8b592-4af7-48fe-9385-0de92afaf07c") : secret "dns-default-metrics-tls" not found Jul 02 19:12:35.742519 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:35.742483 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-x6qcl\" (UID: \"535b962d-6d94-4e02-bd83-4411a9f5fd05\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:12:35.742679 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:35.742633 2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 19:12:35.742719 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:12:35.742699 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert podName:535b962d-6d94-4e02-bd83-4411a9f5fd05 nodeName:}" failed. No retries permitted until 2026-07-02 19:13:07.742684073 +0000 UTC m=+100.289409059 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-x6qcl" (UID: "535b962d-6d94-4e02-bd83-4411a9f5fd05") : secret "networking-console-plugin-cert" not found Jul 02 19:12:40.283437 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:40.283348 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q6q7s" event={"ID":"c8e6d112-aeda-4479-9158-3e91f551e95c","Type":"ContainerStarted","Data":"1269e4f9b6e8ae90c40be3f4364613f811a87d9e0f438875128f296e68f492b4"} Jul 02 19:12:40.283840 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:40.283441 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:12:40.284789 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:40.284760 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bhgfb" event={"ID":"882fce17-202b-44a2-9002-efdcb888f969","Type":"ContainerStarted","Data":"1cdf720b91a8c7410a01a128cf75b84ecd21fe578512973adb65e559cd904d5c"} Jul 02 19:12:40.300728 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:40.300687 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-q6q7s" podStartSLOduration=66.648987812 podStartE2EDuration="1m12.30067369s" totalImagesPullingTime="5.651685878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:40.299638304 +0000 UTC m=+72.846363328" watchObservedRunningTime="2026-07-02 19:12:40.30067369 +0000 UTC m=+72.847398856" Jul 02 19:12:40.316360 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:40.316322 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bhgfb" podStartSLOduration=66.431356999 podStartE2EDuration="1m12.316307652s" totalImagesPullingTime="5.884950653s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:40.315503223 +0000 UTC m=+72.862228251" watchObservedRunningTime="2026-07-02 19:12:40.316307652 +0000 UTC m=+72.863032658" Jul 02 19:12:46.512447 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.512414 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88"] Jul 02 19:12:46.514726 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.514709 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n"] Jul 02 19:12:46.514852 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.514832 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" Jul 02 19:12:46.516574 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.516556 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.518594 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.518570 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 02 19:12:46.518709 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.518695 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 02 19:12:46.518778 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.518707 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 02 19:12:46.519634 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.519618 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 02 19:12:46.522783 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.522761 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 02 19:12:46.522783 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.522780 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 02 19:12:46.522930 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.522771 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 02 19:12:46.522930 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.522829 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 02 19:12:46.529091 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.529072 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n"] Jul 02 19:12:46.532477 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.532459 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88"] Jul 02 19:12:46.619289 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.619259 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/979002b9-e8bb-4d90-a8ee-0414929b2abb-klusterlet-config\") pod \"klusterlet-addon-workmgr-579c7bb775-zhh88\" (UID: \"979002b9-e8bb-4d90-a8ee-0414929b2abb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" Jul 02 19:12:46.619459 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.619303 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2ac04474-e029-4591-9232-ec7fee4ee329-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-68c4cc8d98-g6l8n\" (UID: \"2ac04474-e029-4591-9232-ec7fee4ee329\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.619459 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.619323 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/979002b9-e8bb-4d90-a8ee-0414929b2abb-tmp\") pod \"klusterlet-addon-workmgr-579c7bb775-zhh88\" (UID: \"979002b9-e8bb-4d90-a8ee-0414929b2abb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" Jul 02 19:12:46.619459 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.619338 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2ac04474-e029-4591-9232-ec7fee4ee329-hub\") pod \"cluster-proxy-proxy-agent-68c4cc8d98-g6l8n\" (UID: \"2ac04474-e029-4591-9232-ec7fee4ee329\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.619459 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.619381 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb2dw\" (UniqueName: \"kubernetes.io/projected/979002b9-e8bb-4d90-a8ee-0414929b2abb-kube-api-access-mb2dw\") pod \"klusterlet-addon-workmgr-579c7bb775-zhh88\" (UID: \"979002b9-e8bb-4d90-a8ee-0414929b2abb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" Jul 02 19:12:46.619459 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.619423 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2ac04474-e029-4591-9232-ec7fee4ee329-ca\") pod \"cluster-proxy-proxy-agent-68c4cc8d98-g6l8n\" (UID: \"2ac04474-e029-4591-9232-ec7fee4ee329\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.619459 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.619447 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2ac04474-e029-4591-9232-ec7fee4ee329-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-68c4cc8d98-g6l8n\" (UID: \"2ac04474-e029-4591-9232-ec7fee4ee329\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.619459 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.619462 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9tkq\" (UniqueName: \"kubernetes.io/projected/2ac04474-e029-4591-9232-ec7fee4ee329-kube-api-access-c9tkq\") pod \"cluster-proxy-proxy-agent-68c4cc8d98-g6l8n\" (UID: \"2ac04474-e029-4591-9232-ec7fee4ee329\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.619747 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.619574 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac04474-e029-4591-9232-ec7fee4ee329-additional-service-ca\") pod \"cluster-proxy-proxy-agent-68c4cc8d98-g6l8n\" (UID: \"2ac04474-e029-4591-9232-ec7fee4ee329\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.720468 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.720430 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/979002b9-e8bb-4d90-a8ee-0414929b2abb-klusterlet-config\") pod \"klusterlet-addon-workmgr-579c7bb775-zhh88\" (UID: \"979002b9-e8bb-4d90-a8ee-0414929b2abb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" Jul 02 19:12:46.720631 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.720486 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2ac04474-e029-4591-9232-ec7fee4ee329-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-68c4cc8d98-g6l8n\" (UID: \"2ac04474-e029-4591-9232-ec7fee4ee329\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.720696 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.720648 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/979002b9-e8bb-4d90-a8ee-0414929b2abb-tmp\") pod \"klusterlet-addon-workmgr-579c7bb775-zhh88\" (UID: \"979002b9-e8bb-4d90-a8ee-0414929b2abb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" Jul 02 19:12:46.720696 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.720673 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2ac04474-e029-4591-9232-ec7fee4ee329-hub\") pod \"cluster-proxy-proxy-agent-68c4cc8d98-g6l8n\" (UID: \"2ac04474-e029-4591-9232-ec7fee4ee329\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.720802 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.720694 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb2dw\" (UniqueName: \"kubernetes.io/projected/979002b9-e8bb-4d90-a8ee-0414929b2abb-kube-api-access-mb2dw\") pod \"klusterlet-addon-workmgr-579c7bb775-zhh88\" (UID: \"979002b9-e8bb-4d90-a8ee-0414929b2abb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" Jul 02 19:12:46.720802 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.720717 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2ac04474-e029-4591-9232-ec7fee4ee329-ca\") pod \"cluster-proxy-proxy-agent-68c4cc8d98-g6l8n\" (UID: \"2ac04474-e029-4591-9232-ec7fee4ee329\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.720802 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.720742 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2ac04474-e029-4591-9232-ec7fee4ee329-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-68c4cc8d98-g6l8n\" (UID: \"2ac04474-e029-4591-9232-ec7fee4ee329\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.720802 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.720769 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9tkq\" (UniqueName: \"kubernetes.io/projected/2ac04474-e029-4591-9232-ec7fee4ee329-kube-api-access-c9tkq\") pod \"cluster-proxy-proxy-agent-68c4cc8d98-g6l8n\" (UID: \"2ac04474-e029-4591-9232-ec7fee4ee329\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.721009 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.720847 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac04474-e029-4591-9232-ec7fee4ee329-additional-service-ca\") pod \"cluster-proxy-proxy-agent-68c4cc8d98-g6l8n\" (UID: \"2ac04474-e029-4591-9232-ec7fee4ee329\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.721080 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.721058 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/979002b9-e8bb-4d90-a8ee-0414929b2abb-tmp\") pod \"klusterlet-addon-workmgr-579c7bb775-zhh88\" (UID: \"979002b9-e8bb-4d90-a8ee-0414929b2abb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" Jul 02 19:12:46.721596 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.721562 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac04474-e029-4591-9232-ec7fee4ee329-additional-service-ca\") pod \"cluster-proxy-proxy-agent-68c4cc8d98-g6l8n\" (UID: \"2ac04474-e029-4591-9232-ec7fee4ee329\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.723188 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.723163 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2ac04474-e029-4591-9232-ec7fee4ee329-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-68c4cc8d98-g6l8n\" (UID: \"2ac04474-e029-4591-9232-ec7fee4ee329\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.723345 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.723316 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2ac04474-e029-4591-9232-ec7fee4ee329-hub\") pod \"cluster-proxy-proxy-agent-68c4cc8d98-g6l8n\" (UID: \"2ac04474-e029-4591-9232-ec7fee4ee329\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.723388 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.723375 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2ac04474-e029-4591-9232-ec7fee4ee329-ca\") pod \"cluster-proxy-proxy-agent-68c4cc8d98-g6l8n\" (UID: \"2ac04474-e029-4591-9232-ec7fee4ee329\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.723448 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.723426 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2ac04474-e029-4591-9232-ec7fee4ee329-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-68c4cc8d98-g6l8n\" (UID: \"2ac04474-e029-4591-9232-ec7fee4ee329\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.723531 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.723517 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/979002b9-e8bb-4d90-a8ee-0414929b2abb-klusterlet-config\") pod \"klusterlet-addon-workmgr-579c7bb775-zhh88\" (UID: \"979002b9-e8bb-4d90-a8ee-0414929b2abb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" Jul 02 19:12:46.729866 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.729840 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9tkq\" (UniqueName: \"kubernetes.io/projected/2ac04474-e029-4591-9232-ec7fee4ee329-kube-api-access-c9tkq\") pod \"cluster-proxy-proxy-agent-68c4cc8d98-g6l8n\" (UID: \"2ac04474-e029-4591-9232-ec7fee4ee329\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.731546 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.731527 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb2dw\" (UniqueName: \"kubernetes.io/projected/979002b9-e8bb-4d90-a8ee-0414929b2abb-kube-api-access-mb2dw\") pod \"klusterlet-addon-workmgr-579c7bb775-zhh88\" (UID: \"979002b9-e8bb-4d90-a8ee-0414929b2abb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" Jul 02 19:12:46.825670 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.825584 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" Jul 02 19:12:46.841213 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.841188 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:12:46.954057 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.954028 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88"] Jul 02 19:12:46.956748 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:12:46.956724 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod979002b9_e8bb_4d90_a8ee_0414929b2abb.slice/crio-e716f1da063bb952e2351d12fd4fb02e6ff9edbed07ba0f0d31da6d8164dcf92 WatchSource:0}: Error finding container e716f1da063bb952e2351d12fd4fb02e6ff9edbed07ba0f0d31da6d8164dcf92: Status 404 returned error can't find the container with id e716f1da063bb952e2351d12fd4fb02e6ff9edbed07ba0f0d31da6d8164dcf92 Jul 02 19:12:46.974987 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:46.974963 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n"] Jul 02 19:12:46.977821 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:12:46.977793 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ac04474_e029_4591_9232_ec7fee4ee329.slice/crio-4432c961469ed97c0b7b17293c11cbe0292618f785c024634477bbcb1e13dcdd WatchSource:0}: Error finding container 4432c961469ed97c0b7b17293c11cbe0292618f785c024634477bbcb1e13dcdd: Status 404 returned error can't find the container with id 4432c961469ed97c0b7b17293c11cbe0292618f785c024634477bbcb1e13dcdd Jul 02 19:12:47.301821 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:47.301782 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" event={"ID":"2ac04474-e029-4591-9232-ec7fee4ee329","Type":"ContainerStarted","Data":"4432c961469ed97c0b7b17293c11cbe0292618f785c024634477bbcb1e13dcdd"} Jul 02 19:12:47.302888 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:47.302860 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" event={"ID":"979002b9-e8bb-4d90-a8ee-0414929b2abb","Type":"ContainerStarted","Data":"e716f1da063bb952e2351d12fd4fb02e6ff9edbed07ba0f0d31da6d8164dcf92"} Jul 02 19:12:51.315737 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:51.315701 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" event={"ID":"2ac04474-e029-4591-9232-ec7fee4ee329","Type":"ContainerStarted","Data":"c7362bb60f251989432a1b7a5d3b09bd3b4434765f391ac7c5eac2c4c5f2d340"} Jul 02 19:12:51.316125 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:51.315748 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" event={"ID":"2ac04474-e029-4591-9232-ec7fee4ee329","Type":"ContainerStarted","Data":"742079ee5acb3c63083af11d95b2657be58cf7cd2b4173525c28731ced7deace"} Jul 02 19:12:51.316125 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:51.315767 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" event={"ID":"2ac04474-e029-4591-9232-ec7fee4ee329","Type":"ContainerStarted","Data":"5e4557d387fa8687da55d3339348cec9844a9b4685d5e3dc11fa884b79c2b548"} Jul 02 19:12:51.335474 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:51.335424 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" podStartSLOduration=1.74355648 podStartE2EDuration="5.335409559s" totalImagesPullingTime="3.591853079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:51.334072318 +0000 UTC m=+83.880797334" watchObservedRunningTime="2026-07-02 19:12:51.335409559 +0000 UTC m=+83.882134565" Jul 02 19:12:56.336888 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:56.336792 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" event={"ID":"979002b9-e8bb-4d90-a8ee-0414929b2abb","Type":"ContainerStarted","Data":"9e80d29cd995d4056a7eabf4c025851ebed410f024c3463de1d30b04c8606fd0"} Jul 02 19:12:56.337353 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:56.336991 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" Jul 02 19:12:56.338556 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:56.338534 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" Jul 02 19:12:56.353982 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:12:56.353936 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" podStartSLOduration=1.382609658 podStartE2EDuration="10.353923298s" totalImagesPullingTime="8.97131364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:56.352823614 +0000 UTC m=+88.899548622" watchObservedRunningTime="2026-07-02 19:12:56.353923298 +0000 UTC m=+88.900648305" Jul 02 19:13:07.687238 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:13:07.687199 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:13:07.687658 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:13:07.687247 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:13:07.687658 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:13:07.687345 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:13:07.687658 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:13:07.687365 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 19:13:07.687658 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:13:07.687375 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cddd996cc-6qvxb: secret "image-registry-tls" not found Jul 02 19:13:07.687658 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:13:07.687410 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls podName:77f8b592-4af7-48fe-9385-0de92afaf07c nodeName:}" failed. No retries permitted until 2026-07-02 19:14:11.6873938 +0000 UTC m=+164.234118785 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls") pod "dns-default-7qq96" (UID: "77f8b592-4af7-48fe-9385-0de92afaf07c") : secret "dns-default-metrics-tls" not found Jul 02 19:13:07.687658 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:13:07.687425 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls podName:768126ea-6b59-47d9-a137-b71cd0695d33 nodeName:}" failed. No retries permitted until 2026-07-02 19:14:11.687419142 +0000 UTC m=+164.234144127 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls") pod "image-registry-6cddd996cc-6qvxb" (UID: "768126ea-6b59-47d9-a137-b71cd0695d33") : secret "image-registry-tls" not found Jul 02 19:13:07.788125 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:13:07.788082 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-x6qcl\" (UID: \"535b962d-6d94-4e02-bd83-4411a9f5fd05\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:13:07.788302 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:13:07.788220 2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 19:13:07.788302 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:13:07.788272 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert podName:535b962d-6d94-4e02-bd83-4411a9f5fd05 nodeName:}" failed. No retries permitted until 2026-07-02 19:14:11.788259197 +0000 UTC m=+164.334984181 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-x6qcl" (UID: "535b962d-6d94-4e02-bd83-4411a9f5fd05") : secret "networking-console-plugin-cert" not found Jul 02 19:13:11.289204 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:13:11.289169 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-q6q7s" Jul 02 19:13:37.713770 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:13:37.713728 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs\") pod \"network-metrics-daemon-pjsd8\" (UID: \"75464ead-f0fb-4ea8-beba-55b639087860\") " pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:13:37.714274 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:13:37.713876 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 02 19:13:37.714274 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:13:37.713952 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs podName:75464ead-f0fb-4ea8-beba-55b639087860 nodeName:}" failed. No retries permitted until 2026-07-02 19:15:39.713934729 +0000 UTC m=+252.260659713 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs") pod "network-metrics-daemon-pjsd8" (UID: "75464ead-f0fb-4ea8-beba-55b639087860") : secret "metrics-daemon-secret" not found Jul 02 19:14:06.783315 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:14:06.783225 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" podUID="768126ea-6b59-47d9-a137-b71cd0695d33" Jul 02 19:14:06.806636 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:14:06.806607 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7qq96" podUID="77f8b592-4af7-48fe-9385-0de92afaf07c" Jul 02 19:14:06.852786 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:14:06.852748 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" podUID="535b962d-6d94-4e02-bd83-4411a9f5fd05" Jul 02 19:14:07.539627 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:14:07.539596 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:14:07.539789 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:14:07.539596 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7qq96" Jul 02 19:14:07.983915 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:14:07.983881 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:14:11.771907 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:14:11.771871 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:14:11.772388 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:14:11.771918 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:14:11.772388 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:14:11.772029 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:14:11.772388 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:14:11.772083 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls podName:77f8b592-4af7-48fe-9385-0de92afaf07c nodeName:}" failed. No retries permitted until 2026-07-02 19:16:13.772069286 +0000 UTC m=+286.318794271 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls") pod "dns-default-7qq96" (UID: "77f8b592-4af7-48fe-9385-0de92afaf07c") : secret "dns-default-metrics-tls" not found Jul 02 19:14:11.772388 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:14:11.772090 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 19:14:11.772388 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:14:11.772110 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cddd996cc-6qvxb: secret "image-registry-tls" not found Jul 02 19:14:11.772388 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:14:11.772177 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls podName:768126ea-6b59-47d9-a137-b71cd0695d33 nodeName:}" failed. No retries permitted until 2026-07-02 19:16:13.772164462 +0000 UTC m=+286.318889460 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls") pod "image-registry-6cddd996cc-6qvxb" (UID: "768126ea-6b59-47d9-a137-b71cd0695d33") : secret "image-registry-tls" not found Jul 02 19:14:11.872933 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:14:11.872889 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-x6qcl\" (UID: \"535b962d-6d94-4e02-bd83-4411a9f5fd05\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:14:11.873045 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:14:11.873028 2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 19:14:11.873098 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:14:11.873089 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert podName:535b962d-6d94-4e02-bd83-4411a9f5fd05 nodeName:}" failed. No retries permitted until 2026-07-02 19:16:13.873073209 +0000 UTC m=+286.419798198 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-x6qcl" (UID: "535b962d-6d94-4e02-bd83-4411a9f5fd05") : secret "networking-console-plugin-cert" not found Jul 02 19:14:17.973346 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:14:17.973314 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:14:22.973127 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:14:22.973083 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:14:56.337867 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:14:56.337753 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" podUID="979002b9-e8bb-4d90-a8ee-0414929b2abb" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.11:8000/readyz\": dial tcp 10.132.0.11:8000: connect: connection refused" Jul 02 19:14:56.684519 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:14:56.684488 2556 generic.go:356] "Generic (PLEG): container finished" podID="979002b9-e8bb-4d90-a8ee-0414929b2abb" containerID="9e80d29cd995d4056a7eabf4c025851ebed410f024c3463de1d30b04c8606fd0" exitCode=1 Jul 02 19:14:56.684671 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:14:56.684551 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" event={"ID":"979002b9-e8bb-4d90-a8ee-0414929b2abb","Type":"ContainerDied","Data":"9e80d29cd995d4056a7eabf4c025851ebed410f024c3463de1d30b04c8606fd0"} Jul 02 19:14:56.684854 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:14:56.684841 2556 scope.go:117] "RemoveContainer" containerID="9e80d29cd995d4056a7eabf4c025851ebed410f024c3463de1d30b04c8606fd0" Jul 02 19:14:56.826701 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:14:56.826670 2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" Jul 02 19:14:57.689015 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:14:57.688980 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" event={"ID":"979002b9-e8bb-4d90-a8ee-0414929b2abb","Type":"ContainerStarted","Data":"28b8598e62189c21ea6c9d032828c54cacd4f94042f94a3e9ee61465217259c9"} Jul 02 19:14:57.689514 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:14:57.689275 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" Jul 02 19:14:57.690208 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:14:57.690191 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579c7bb775-zhh88" Jul 02 19:15:39.795469 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:15:39.795420 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs\") pod \"network-metrics-daemon-pjsd8\" (UID: \"75464ead-f0fb-4ea8-beba-55b639087860\") " pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:15:39.795968 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:15:39.795542 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 02 19:15:39.795968 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:15:39.795591 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs podName:75464ead-f0fb-4ea8-beba-55b639087860 nodeName:}" failed. No retries permitted until 2026-07-02 19:17:41.795578781 +0000 UTC m=+374.342303767 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs") pod "network-metrics-daemon-pjsd8" (UID: "75464ead-f0fb-4ea8-beba-55b639087860") : secret "metrics-daemon-secret" not found Jul 02 19:16:02.717118 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.717040 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-g99mz"] Jul 02 19:16:02.719993 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.719975 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:02.723434 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.723411 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 19:16:02.724033 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.724007 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9h8cm\"" Jul 02 19:16:02.724134 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.724035 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 19:16:02.724134 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.724036 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 19:16:02.724134 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.724014 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 19:16:02.743507 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.743469 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g99mz"] Jul 02 19:16:02.869592 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.869554 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/608ae54a-8df6-4349-8d64-4fa26c784657-crio-socket\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:02.869765 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.869601 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/608ae54a-8df6-4349-8d64-4fa26c784657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:02.869765 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.869674 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/608ae54a-8df6-4349-8d64-4fa26c784657-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:02.869765 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.869716 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/608ae54a-8df6-4349-8d64-4fa26c784657-data-volume\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:02.869765 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.869757 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqn5m\" (UniqueName: \"kubernetes.io/projected/608ae54a-8df6-4349-8d64-4fa26c784657-kube-api-access-tqn5m\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:02.970129 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.970038 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/608ae54a-8df6-4349-8d64-4fa26c784657-crio-socket\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:02.970129 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.970092 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/608ae54a-8df6-4349-8d64-4fa26c784657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:02.970371 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.970181 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/608ae54a-8df6-4349-8d64-4fa26c784657-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:02.970371 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.970227 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/608ae54a-8df6-4349-8d64-4fa26c784657-data-volume\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:02.970371 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.970230 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/608ae54a-8df6-4349-8d64-4fa26c784657-crio-socket\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:02.970371 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.970278 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqn5m\" (UniqueName: \"kubernetes.io/projected/608ae54a-8df6-4349-8d64-4fa26c784657-kube-api-access-tqn5m\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:02.970371 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:02.970279 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 19:16:02.970371 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:02.970308 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 19:16:02.970371 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:02.970336 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/608ae54a-8df6-4349-8d64-4fa26c784657-insights-runtime-extractor-tls podName:608ae54a-8df6-4349-8d64-4fa26c784657 nodeName:}" failed. No retries permitted until 2026-07-02 19:16:03.470320593 +0000 UTC m=+276.017045577 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/608ae54a-8df6-4349-8d64-4fa26c784657-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g99mz" (UID: "608ae54a-8df6-4349-8d64-4fa26c784657") : secret "insights-runtime-extractor-tls" not found Jul 02 19:16:02.970626 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:02.970392 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/608ae54a-8df6-4349-8d64-4fa26c784657-kube-rbac-proxy-cm podName:608ae54a-8df6-4349-8d64-4fa26c784657 nodeName:}" failed. No retries permitted until 2026-07-02 19:16:03.470374468 +0000 UTC m=+276.017099460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/608ae54a-8df6-4349-8d64-4fa26c784657-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g99mz" (UID: "608ae54a-8df6-4349-8d64-4fa26c784657") : configmap "kube-rbac-proxy" not found Jul 02 19:16:02.970626 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.970612 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/608ae54a-8df6-4349-8d64-4fa26c784657-data-volume\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:02.981817 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:02.981794 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqn5m\" (UniqueName: \"kubernetes.io/projected/608ae54a-8df6-4349-8d64-4fa26c784657-kube-api-access-tqn5m\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:03.474231 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:03.474186 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/608ae54a-8df6-4349-8d64-4fa26c784657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:03.474405 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:03.474256 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/608ae54a-8df6-4349-8d64-4fa26c784657-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:03.474405 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:03.474326 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 19:16:03.474405 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:03.474376 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/608ae54a-8df6-4349-8d64-4fa26c784657-kube-rbac-proxy-cm podName:608ae54a-8df6-4349-8d64-4fa26c784657 nodeName:}" failed. No retries permitted until 2026-07-02 19:16:04.474362455 +0000 UTC m=+277.021087441 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/608ae54a-8df6-4349-8d64-4fa26c784657-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g99mz" (UID: "608ae54a-8df6-4349-8d64-4fa26c784657") : configmap "kube-rbac-proxy" not found Jul 02 19:16:03.474405 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:03.474327 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 19:16:03.474543 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:03.474468 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/608ae54a-8df6-4349-8d64-4fa26c784657-insights-runtime-extractor-tls podName:608ae54a-8df6-4349-8d64-4fa26c784657 nodeName:}" failed. No retries permitted until 2026-07-02 19:16:04.474453043 +0000 UTC m=+277.021178027 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/608ae54a-8df6-4349-8d64-4fa26c784657-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g99mz" (UID: "608ae54a-8df6-4349-8d64-4fa26c784657") : secret "insights-runtime-extractor-tls" not found Jul 02 19:16:04.483204 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:04.483170 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/608ae54a-8df6-4349-8d64-4fa26c784657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:04.483575 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:04.483231 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/608ae54a-8df6-4349-8d64-4fa26c784657-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:04.483575 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:04.483324 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 19:16:04.483575 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:04.483373 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 19:16:04.483575 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:04.483383 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/608ae54a-8df6-4349-8d64-4fa26c784657-insights-runtime-extractor-tls podName:608ae54a-8df6-4349-8d64-4fa26c784657 nodeName:}" failed. No retries permitted until 2026-07-02 19:16:06.483369034 +0000 UTC m=+279.030094039 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/608ae54a-8df6-4349-8d64-4fa26c784657-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g99mz" (UID: "608ae54a-8df6-4349-8d64-4fa26c784657") : secret "insights-runtime-extractor-tls" not found Jul 02 19:16:04.483575 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:04.483445 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/608ae54a-8df6-4349-8d64-4fa26c784657-kube-rbac-proxy-cm podName:608ae54a-8df6-4349-8d64-4fa26c784657 nodeName:}" failed. No retries permitted until 2026-07-02 19:16:06.483429187 +0000 UTC m=+279.030154172 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/608ae54a-8df6-4349-8d64-4fa26c784657-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g99mz" (UID: "608ae54a-8df6-4349-8d64-4fa26c784657") : configmap "kube-rbac-proxy" not found Jul 02 19:16:04.902324 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:04.902249 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sv2qm_dee24e3b-8c4a-48d3-9c21-31c9ca499d72/dns-node-resolver/0.log" Jul 02 19:16:05.702417 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:05.702390 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p8crb_205e2e72-bfab-43b6-b8d3-8af9a1c8561a/node-ca/0.log" Jul 02 19:16:06.497424 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:06.497391 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/608ae54a-8df6-4349-8d64-4fa26c784657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:06.497606 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:06.497449 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/608ae54a-8df6-4349-8d64-4fa26c784657-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:06.497606 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:06.497520 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 19:16:06.497606 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:06.497560 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 19:16:06.497606 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:06.497576 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/608ae54a-8df6-4349-8d64-4fa26c784657-kube-rbac-proxy-cm podName:608ae54a-8df6-4349-8d64-4fa26c784657 nodeName:}" failed. No retries permitted until 2026-07-02 19:16:10.497560713 +0000 UTC m=+283.044285699 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/608ae54a-8df6-4349-8d64-4fa26c784657-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g99mz" (UID: "608ae54a-8df6-4349-8d64-4fa26c784657") : configmap "kube-rbac-proxy" not found Jul 02 19:16:06.497745 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:06.497629 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/608ae54a-8df6-4349-8d64-4fa26c784657-insights-runtime-extractor-tls podName:608ae54a-8df6-4349-8d64-4fa26c784657 nodeName:}" failed. No retries permitted until 2026-07-02 19:16:10.497612337 +0000 UTC m=+283.044337322 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/608ae54a-8df6-4349-8d64-4fa26c784657-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g99mz" (UID: "608ae54a-8df6-4349-8d64-4fa26c784657") : secret "insights-runtime-extractor-tls" not found Jul 02 19:16:06.506010 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:06.505987 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8t8d8_171c244d-2dda-414a-a89f-c56e1a47c97c/serve-healthcheck-canary/0.log" Jul 02 19:16:10.524924 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:10.524883 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/608ae54a-8df6-4349-8d64-4fa26c784657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:10.525419 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:10.524962 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/608ae54a-8df6-4349-8d64-4fa26c784657-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:10.525419 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:10.525047 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 19:16:10.525419 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:10.525060 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 19:16:10.525419 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:10.525123 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/608ae54a-8df6-4349-8d64-4fa26c784657-insights-runtime-extractor-tls podName:608ae54a-8df6-4349-8d64-4fa26c784657 nodeName:}" failed. No retries permitted until 2026-07-02 19:16:18.525104918 +0000 UTC m=+291.071829913 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/608ae54a-8df6-4349-8d64-4fa26c784657-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g99mz" (UID: "608ae54a-8df6-4349-8d64-4fa26c784657") : secret "insights-runtime-extractor-tls" not found Jul 02 19:16:10.525419 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:10.525142 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/608ae54a-8df6-4349-8d64-4fa26c784657-kube-rbac-proxy-cm podName:608ae54a-8df6-4349-8d64-4fa26c784657 nodeName:}" failed. No retries permitted until 2026-07-02 19:16:18.525133094 +0000 UTC m=+291.071858082 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/608ae54a-8df6-4349-8d64-4fa26c784657-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g99mz" (UID: "608ae54a-8df6-4349-8d64-4fa26c784657") : configmap "kube-rbac-proxy" not found Jul 02 19:16:10.540868 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:10.540836 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" podUID="535b962d-6d94-4e02-bd83-4411a9f5fd05" Jul 02 19:16:10.540995 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:10.540836 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7qq96" podUID="77f8b592-4af7-48fe-9385-0de92afaf07c" Jul 02 19:16:10.892173 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:10.892075 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:16:10.892173 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:10.892075 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7qq96" Jul 02 19:16:13.851204 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:13.851176 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:16:13.851573 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:13.851216 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:16:13.853420 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:13.853392 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77f8b592-4af7-48fe-9385-0de92afaf07c-metrics-tls\") pod \"dns-default-7qq96\" (UID: \"77f8b592-4af7-48fe-9385-0de92afaf07c\") " pod="openshift-dns/dns-default-7qq96" Jul 02 19:16:13.853554 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:13.853537 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls\") pod \"image-registry-6cddd996cc-6qvxb\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:16:13.895691 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:13.895664 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9snh5\"" Jul 02 19:16:13.903678 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:13.903659 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7qq96" Jul 02 19:16:13.952073 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:13.952037 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-x6qcl\" (UID: \"535b962d-6d94-4e02-bd83-4411a9f5fd05\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:16:13.952243 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:13.952222 2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 19:16:13.952326 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:13.952313 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert podName:535b962d-6d94-4e02-bd83-4411a9f5fd05 nodeName:}" failed. No retries permitted until 2026-07-02 19:18:15.952290134 +0000 UTC m=+408.499015119 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-x6qcl" (UID: "535b962d-6d94-4e02-bd83-4411a9f5fd05") : secret "networking-console-plugin-cert" not found Jul 02 19:16:14.029810 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:14.029777 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7qq96"] Jul 02 19:16:14.032971 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:16:14.032946 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77f8b592_4af7_48fe_9385_0de92afaf07c.slice/crio-d45fb1e9ab3921204eb85620972edb85cd1e96119066a5958f3066dad50f5bd2 WatchSource:0}: Error finding container d45fb1e9ab3921204eb85620972edb85cd1e96119066a5958f3066dad50f5bd2: Status 404 returned error can't find the container with id d45fb1e9ab3921204eb85620972edb85cd1e96119066a5958f3066dad50f5bd2 Jul 02 19:16:14.076755 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:14.076728 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7dhp8\"" Jul 02 19:16:14.084090 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:14.084068 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:16:14.206540 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:14.206506 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cddd996cc-6qvxb"] Jul 02 19:16:14.209599 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:16:14.209573 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod768126ea_6b59_47d9_a137_b71cd0695d33.slice/crio-aa455b050436e12646982b1f336925b99f0a50144705d79db4ede9e8a3d0e54c WatchSource:0}: Error finding container aa455b050436e12646982b1f336925b99f0a50144705d79db4ede9e8a3d0e54c: Status 404 returned error can't find the container with id aa455b050436e12646982b1f336925b99f0a50144705d79db4ede9e8a3d0e54c Jul 02 19:16:14.905946 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:14.905915 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" event={"ID":"768126ea-6b59-47d9-a137-b71cd0695d33","Type":"ContainerStarted","Data":"29a3aefd384da667da1c7cab064015331e96dc9d581cc01c32b34c40d30a8e76"} Jul 02 19:16:14.906466 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:14.905972 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" event={"ID":"768126ea-6b59-47d9-a137-b71cd0695d33","Type":"ContainerStarted","Data":"aa455b050436e12646982b1f336925b99f0a50144705d79db4ede9e8a3d0e54c"} Jul 02 19:16:14.906466 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:14.906038 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:16:14.907035 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:14.907016 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7qq96" event={"ID":"77f8b592-4af7-48fe-9385-0de92afaf07c","Type":"ContainerStarted","Data":"d45fb1e9ab3921204eb85620972edb85cd1e96119066a5958f3066dad50f5bd2"} Jul 02 19:16:14.928320 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:14.928277 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" podStartSLOduration=286.928262791 podStartE2EDuration="4m46.928262791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:11:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:16:14.927252767 +0000 UTC m=+287.473977791" watchObservedRunningTime="2026-07-02 19:16:14.928262791 +0000 UTC m=+287.474987817" Jul 02 19:16:16.914275 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:16.914241 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7qq96" event={"ID":"77f8b592-4af7-48fe-9385-0de92afaf07c","Type":"ContainerStarted","Data":"536310d32d8a34c249385f676344dfb8f8cdf39d358e503f49adcf750d0d2260"} Jul 02 19:16:16.914686 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:16.914289 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7qq96" event={"ID":"77f8b592-4af7-48fe-9385-0de92afaf07c","Type":"ContainerStarted","Data":"8ffce62597c1d3cf16524b7c4f66b5b9bc61e454f073c63c188d57c45575e402"} Jul 02 19:16:16.914686 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:16.914379 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7qq96" Jul 02 19:16:16.934084 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:16.934040 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7qq96" podStartSLOduration=251.762767764 podStartE2EDuration="4m13.934026157s" totalImagesPullingTime="2.171258393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:16:16.932679272 +0000 UTC m=+289.479404278" watchObservedRunningTime="2026-07-02 19:16:16.934026157 +0000 UTC m=+289.480751164" Jul 02 19:16:18.585342 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:18.585300 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/608ae54a-8df6-4349-8d64-4fa26c784657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:18.585719 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:18.585356 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/608ae54a-8df6-4349-8d64-4fa26c784657-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:18.585854 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:18.585836 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/608ae54a-8df6-4349-8d64-4fa26c784657-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:18.587513 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:18.587491 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/608ae54a-8df6-4349-8d64-4fa26c784657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g99mz\" (UID: \"608ae54a-8df6-4349-8d64-4fa26c784657\") " pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:18.629628 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:18.629594 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g99mz" Jul 02 19:16:18.756242 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:18.756210 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g99mz"] Jul 02 19:16:18.759285 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:16:18.759258 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod608ae54a_8df6_4349_8d64_4fa26c784657.slice/crio-a04390bbfe94bd83e191c8e2914ce244b54d78acf246fc11bfbd9d600a08fac4 WatchSource:0}: Error finding container a04390bbfe94bd83e191c8e2914ce244b54d78acf246fc11bfbd9d600a08fac4: Status 404 returned error can't find the container with id a04390bbfe94bd83e191c8e2914ce244b54d78acf246fc11bfbd9d600a08fac4 Jul 02 19:16:18.921873 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:18.921840 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g99mz" event={"ID":"608ae54a-8df6-4349-8d64-4fa26c784657","Type":"ContainerStarted","Data":"a04390bbfe94bd83e191c8e2914ce244b54d78acf246fc11bfbd9d600a08fac4"} Jul 02 19:16:22.936854 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:22.936820 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g99mz" event={"ID":"608ae54a-8df6-4349-8d64-4fa26c784657","Type":"ContainerStarted","Data":"052f4e7f540f7e15474ca18c630158552946d3ecb00452b015ced8a784c434c4"} Jul 02 19:16:23.941582 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:23.941539 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g99mz" event={"ID":"608ae54a-8df6-4349-8d64-4fa26c784657","Type":"ContainerStarted","Data":"6994b52f0a0ad1590167d81336a1b5ae8b7b36f89b3671733e87f1d2c1a550c1"} Jul 02 19:16:25.950021 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:25.949986 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g99mz" event={"ID":"608ae54a-8df6-4349-8d64-4fa26c784657","Type":"ContainerStarted","Data":"5c0f7aa12f3b28397f145a227329b058e8797ed24dfc317de43cc7a64fee57c8"} Jul 02 19:16:25.969403 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:25.969354 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-g99mz" podStartSLOduration=17.304815739 podStartE2EDuration="23.969339444s" totalImagesPullingTime="6.664523705s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:16:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:16:25.967500176 +0000 UTC m=+298.514225182" watchObservedRunningTime="2026-07-02 19:16:25.969339444 +0000 UTC m=+298.516064450" Jul 02 19:16:25.973830 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:25.973805 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-pjsd8" podUID="75464ead-f0fb-4ea8-beba-55b639087860" Jul 02 19:16:26.842878 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:26.842841 2556 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" podUID="2ac04474-e029-4591-9232-ec7fee4ee329" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 02 19:16:26.920377 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:26.920348 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7qq96" Jul 02 19:16:27.836753 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:27.836726 2556 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 19:16:34.088237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:34.088204 2556 patch_prober.go:28] interesting pod/image-registry-6cddd996cc-6qvxb container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 02 19:16:34.090514 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:34.088272 2556 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" podUID="768126ea-6b59-47d9-a137-b71cd0695d33" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 19:16:35.914973 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:35.914943 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:16:36.842449 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:36.842414 2556 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" podUID="2ac04474-e029-4591-9232-ec7fee4ee329" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 02 19:16:39.972954 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:39.972917 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:16:40.008100 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.008070 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ls6fs"] Jul 02 19:16:40.012496 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.012478 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.018808 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.018610 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 19:16:40.018808 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.018671 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 19:16:40.019695 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.019636 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hhr7x\"" Jul 02 19:16:40.019695 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.019641 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 19:16:40.019695 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.019666 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 19:16:40.020051 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.020023 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 19:16:40.020351 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.020335 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 19:16:40.045926 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.045895 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/78caeef4-8166-4bba-b486-d1ba7cd02385-node-exporter-wtmp\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.046078 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.045935 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5hwt\" (UniqueName: \"kubernetes.io/projected/78caeef4-8166-4bba-b486-d1ba7cd02385-kube-api-access-c5hwt\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.046078 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.045992 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/78caeef4-8166-4bba-b486-d1ba7cd02385-node-exporter-textfile\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.046078 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.046022 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78caeef4-8166-4bba-b486-d1ba7cd02385-sys\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.046078 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.046041 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78caeef4-8166-4bba-b486-d1ba7cd02385-metrics-client-ca\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.046078 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.046071 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78caeef4-8166-4bba-b486-d1ba7cd02385-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.046285 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.046097 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/78caeef4-8166-4bba-b486-d1ba7cd02385-root\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.046285 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.046111 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/78caeef4-8166-4bba-b486-d1ba7cd02385-node-exporter-tls\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.046285 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.046125 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/78caeef4-8166-4bba-b486-d1ba7cd02385-node-exporter-accelerators-collector-config\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.147457 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.147418 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/78caeef4-8166-4bba-b486-d1ba7cd02385-node-exporter-wtmp\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.147457 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.147462 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5hwt\" (UniqueName: \"kubernetes.io/projected/78caeef4-8166-4bba-b486-d1ba7cd02385-kube-api-access-c5hwt\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.147668 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.147519 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/78caeef4-8166-4bba-b486-d1ba7cd02385-node-exporter-textfile\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.147668 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.147545 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78caeef4-8166-4bba-b486-d1ba7cd02385-sys\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.147668 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.147570 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78caeef4-8166-4bba-b486-d1ba7cd02385-metrics-client-ca\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.147668 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.147618 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/78caeef4-8166-4bba-b486-d1ba7cd02385-node-exporter-wtmp\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.147668 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.147631 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78caeef4-8166-4bba-b486-d1ba7cd02385-sys\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.147914 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.147709 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78caeef4-8166-4bba-b486-d1ba7cd02385-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.147914 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.147754 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/78caeef4-8166-4bba-b486-d1ba7cd02385-root\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.147914 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.147781 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/78caeef4-8166-4bba-b486-d1ba7cd02385-node-exporter-tls\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.147914 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.147809 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/78caeef4-8166-4bba-b486-d1ba7cd02385-node-exporter-accelerators-collector-config\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.147914 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.147819 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/78caeef4-8166-4bba-b486-d1ba7cd02385-root\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.148191 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:40.147964 2556 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 02 19:16:40.148191 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.147974 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/78caeef4-8166-4bba-b486-d1ba7cd02385-node-exporter-textfile\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.148191 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:16:40.148034 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78caeef4-8166-4bba-b486-d1ba7cd02385-node-exporter-tls podName:78caeef4-8166-4bba-b486-d1ba7cd02385 nodeName:}" failed. No retries permitted until 2026-07-02 19:16:40.64801175 +0000 UTC m=+313.194736734 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/78caeef4-8166-4bba-b486-d1ba7cd02385-node-exporter-tls") pod "node-exporter-ls6fs" (UID: "78caeef4-8166-4bba-b486-d1ba7cd02385") : secret "node-exporter-tls" not found Jul 02 19:16:40.148191 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.148186 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78caeef4-8166-4bba-b486-d1ba7cd02385-metrics-client-ca\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.148405 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.148387 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/78caeef4-8166-4bba-b486-d1ba7cd02385-node-exporter-accelerators-collector-config\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.149974 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.149951 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78caeef4-8166-4bba-b486-d1ba7cd02385-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.159347 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.159326 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5hwt\" (UniqueName: \"kubernetes.io/projected/78caeef4-8166-4bba-b486-d1ba7cd02385-kube-api-access-c5hwt\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.652033 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.651979 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/78caeef4-8166-4bba-b486-d1ba7cd02385-node-exporter-tls\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.654347 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.654323 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/78caeef4-8166-4bba-b486-d1ba7cd02385-node-exporter-tls\") pod \"node-exporter-ls6fs\" (UID: \"78caeef4-8166-4bba-b486-d1ba7cd02385\") " pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.921606 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.921573 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ls6fs" Jul 02 19:16:40.932283 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:16:40.932255 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78caeef4_8166_4bba_b486_d1ba7cd02385.slice/crio-00dfc64521e01236f841987902588ee5264aedeff983f9ef854201edf1bf7a1b WatchSource:0}: Error finding container 00dfc64521e01236f841987902588ee5264aedeff983f9ef854201edf1bf7a1b: Status 404 returned error can't find the container with id 00dfc64521e01236f841987902588ee5264aedeff983f9ef854201edf1bf7a1b Jul 02 19:16:40.934195 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.934176 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 19:16:40.994772 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:40.994734 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ls6fs" event={"ID":"78caeef4-8166-4bba-b486-d1ba7cd02385","Type":"ContainerStarted","Data":"00dfc64521e01236f841987902588ee5264aedeff983f9ef854201edf1bf7a1b"} Jul 02 19:16:43.003932 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:43.003891 2556 generic.go:356] "Generic (PLEG): container finished" podID="78caeef4-8166-4bba-b486-d1ba7cd02385" containerID="2c76028b9be5a2d73b9a729b0fa741d2dd405d468e60a9f01610f6e2d0f771f6" exitCode=0 Jul 02 19:16:43.004373 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:43.003937 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ls6fs" event={"ID":"78caeef4-8166-4bba-b486-d1ba7cd02385","Type":"ContainerDied","Data":"2c76028b9be5a2d73b9a729b0fa741d2dd405d468e60a9f01610f6e2d0f771f6"} Jul 02 19:16:44.008752 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:44.008715 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ls6fs" event={"ID":"78caeef4-8166-4bba-b486-d1ba7cd02385","Type":"ContainerStarted","Data":"0665271ba113350cd8f5805ae7ccf6de271808ebda6eadebb7e2979d2fd3ebd7"} Jul 02 19:16:44.009105 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:44.008773 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ls6fs" event={"ID":"78caeef4-8166-4bba-b486-d1ba7cd02385","Type":"ContainerStarted","Data":"4097f80020e4f097d829a1ca90c38d6dc9da11f95cea9ff2888b922387ec9453"} Jul 02 19:16:44.044616 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:44.044564 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ls6fs" podStartSLOduration=3.973147917 podStartE2EDuration="5.044548454s" totalImagesPullingTime="1.008072323s" totalInitContainerRuntime="63.328214ms" isStatefulPod=true podCreationTimestamp="2026-07-02 19:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:16:44.043544362 +0000 UTC m=+316.590269372" watchObservedRunningTime="2026-07-02 19:16:44.044548454 +0000 UTC m=+316.591273461" Jul 02 19:16:46.842631 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:46.842589 2556 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" podUID="2ac04474-e029-4591-9232-ec7fee4ee329" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 02 19:16:46.843095 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:46.842669 2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" Jul 02 19:16:46.843285 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:46.843249 2556 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"c7362bb60f251989432a1b7a5d3b09bd3b4434765f391ac7c5eac2c4c5f2d340"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 02 19:16:46.843368 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:46.843312 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" podUID="2ac04474-e029-4591-9232-ec7fee4ee329" containerName="service-proxy" containerID="cri-o://c7362bb60f251989432a1b7a5d3b09bd3b4434765f391ac7c5eac2c4c5f2d340" gracePeriod=30 Jul 02 19:16:47.020476 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:47.020443 2556 generic.go:356] "Generic (PLEG): container finished" podID="2ac04474-e029-4591-9232-ec7fee4ee329" containerID="c7362bb60f251989432a1b7a5d3b09bd3b4434765f391ac7c5eac2c4c5f2d340" exitCode=2 Jul 02 19:16:47.020626 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:47.020503 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" event={"ID":"2ac04474-e029-4591-9232-ec7fee4ee329","Type":"ContainerDied","Data":"c7362bb60f251989432a1b7a5d3b09bd3b4434765f391ac7c5eac2c4c5f2d340"} Jul 02 19:16:48.026301 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:48.026263 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68c4cc8d98-g6l8n" event={"ID":"2ac04474-e029-4591-9232-ec7fee4ee329","Type":"ContainerStarted","Data":"4a3ab51cb7e3c68c53397c0d088135a24fee6af06bc909c61332e3c314c47cd1"} Jul 02 19:16:50.056105 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:16:50.056076 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cddd996cc-6qvxb"] Jul 02 19:17:01.484914 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.484883 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6757d494d-7dqvd"] Jul 02 19:17:01.486874 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.486859 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.489589 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.489567 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 02 19:17:01.489710 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.489576 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 02 19:17:01.489710 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.489603 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 02 19:17:01.490858 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.490833 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 02 19:17:01.490974 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.490885 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 02 19:17:01.490974 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.490894 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-m6c26\"" Jul 02 19:17:01.490974 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.490834 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 02 19:17:01.490974 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.490913 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 02 19:17:01.496180 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.496162 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 02 19:17:01.499345 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.499323 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6757d494d-7dqvd"] Jul 02 19:17:01.630571 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.630524 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-oauth-serving-cert\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.630742 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.630592 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-service-ca\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.630742 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.630632 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98dsl\" (UniqueName: \"kubernetes.io/projected/0efd7e51-17b3-406e-85e0-bc1237d2452b-kube-api-access-98dsl\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.630742 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.630658 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0efd7e51-17b3-406e-85e0-bc1237d2452b-console-oauth-config\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.630742 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.630693 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0efd7e51-17b3-406e-85e0-bc1237d2452b-console-serving-cert\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.630742 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.630729 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-console-config\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.630912 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.630799 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-trusted-ca-bundle\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.731900 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.731858 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-trusted-ca-bundle\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.732107 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.731920 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-oauth-serving-cert\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.732107 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.731958 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-service-ca\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.732107 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.732001 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98dsl\" (UniqueName: \"kubernetes.io/projected/0efd7e51-17b3-406e-85e0-bc1237d2452b-kube-api-access-98dsl\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.732107 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.732039 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0efd7e51-17b3-406e-85e0-bc1237d2452b-console-oauth-config\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.732107 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.732064 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0efd7e51-17b3-406e-85e0-bc1237d2452b-console-serving-cert\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.732107 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.732104 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-console-config\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.732796 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.732770 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-oauth-serving-cert\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.732962 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.732880 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-trusted-ca-bundle\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.732962 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.732895 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-console-config\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.733090 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.732937 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-service-ca\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.734543 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.734528 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0efd7e51-17b3-406e-85e0-bc1237d2452b-console-oauth-config\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.734679 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.734663 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0efd7e51-17b3-406e-85e0-bc1237d2452b-console-serving-cert\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.740821 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.740764 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98dsl\" (UniqueName: \"kubernetes.io/projected/0efd7e51-17b3-406e-85e0-bc1237d2452b-kube-api-access-98dsl\") pod \"console-6757d494d-7dqvd\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.797085 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.797048 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:01.925433 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:01.925403 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6757d494d-7dqvd"] Jul 02 19:17:01.928518 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:17:01.928492 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0efd7e51_17b3_406e_85e0_bc1237d2452b.slice/crio-0659de95b4404d5b42b82586b71513cad4d7dbc96fd1934442cd5dd3e8ab098e WatchSource:0}: Error finding container 0659de95b4404d5b42b82586b71513cad4d7dbc96fd1934442cd5dd3e8ab098e: Status 404 returned error can't find the container with id 0659de95b4404d5b42b82586b71513cad4d7dbc96fd1934442cd5dd3e8ab098e Jul 02 19:17:02.069341 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:02.069251 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6757d494d-7dqvd" event={"ID":"0efd7e51-17b3-406e-85e0-bc1237d2452b","Type":"ContainerStarted","Data":"0659de95b4404d5b42b82586b71513cad4d7dbc96fd1934442cd5dd3e8ab098e"} Jul 02 19:17:06.083000 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:06.082966 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6757d494d-7dqvd" event={"ID":"0efd7e51-17b3-406e-85e0-bc1237d2452b","Type":"ContainerStarted","Data":"85afa0eb246ec718c6c43dccdbeeaf34431aa418aecb44e65c220c1d30ac7795"} Jul 02 19:17:06.103662 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:06.103586 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6757d494d-7dqvd" podStartSLOduration=1.8176482790000001 podStartE2EDuration="5.103572373s" totalImagesPullingTime="3.285924094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:17:06.103007185 +0000 UTC m=+338.649732250" watchObservedRunningTime="2026-07-02 19:17:06.103572373 +0000 UTC m=+338.650297423" Jul 02 19:17:11.798176 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:11.798126 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:11.798662 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:11.798200 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:11.802800 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:11.802778 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:12.104542 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:12.104466 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:17:15.075312 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.075277 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" podUID="768126ea-6b59-47d9-a137-b71cd0695d33" containerName="registry" containerID="cri-o://29a3aefd384da667da1c7cab064015331e96dc9d581cc01c32b34c40d30a8e76" gracePeriod=30 Jul 02 19:17:15.325286 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.325259 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:17:15.344861 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.344806 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66zt7\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-kube-api-access-66zt7\") pod \"768126ea-6b59-47d9-a137-b71cd0695d33\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " Jul 02 19:17:15.344861 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.344849 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/768126ea-6b59-47d9-a137-b71cd0695d33-image-registry-private-configuration\") pod \"768126ea-6b59-47d9-a137-b71cd0695d33\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " Jul 02 19:17:15.345025 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.344886 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls\") pod \"768126ea-6b59-47d9-a137-b71cd0695d33\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " Jul 02 19:17:15.345025 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.344912 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/768126ea-6b59-47d9-a137-b71cd0695d33-trusted-ca\") pod \"768126ea-6b59-47d9-a137-b71cd0695d33\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " Jul 02 19:17:15.345025 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.344929 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/768126ea-6b59-47d9-a137-b71cd0695d33-ca-trust-extracted\") pod \"768126ea-6b59-47d9-a137-b71cd0695d33\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " Jul 02 19:17:15.345025 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.344957 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/768126ea-6b59-47d9-a137-b71cd0695d33-registry-certificates\") pod \"768126ea-6b59-47d9-a137-b71cd0695d33\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " Jul 02 19:17:15.345262 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.345036 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-bound-sa-token\") pod \"768126ea-6b59-47d9-a137-b71cd0695d33\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " Jul 02 19:17:15.345262 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.345118 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/768126ea-6b59-47d9-a137-b71cd0695d33-installation-pull-secrets\") pod \"768126ea-6b59-47d9-a137-b71cd0695d33\" (UID: \"768126ea-6b59-47d9-a137-b71cd0695d33\") " Jul 02 19:17:15.345492 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.345433 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/768126ea-6b59-47d9-a137-b71cd0695d33-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "768126ea-6b59-47d9-a137-b71cd0695d33" (UID: "768126ea-6b59-47d9-a137-b71cd0695d33"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:17:15.345492 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.345449 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/768126ea-6b59-47d9-a137-b71cd0695d33-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "768126ea-6b59-47d9-a137-b71cd0695d33" (UID: "768126ea-6b59-47d9-a137-b71cd0695d33"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:17:15.347427 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.347396 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-kube-api-access-66zt7" (OuterVolumeSpecName: "kube-api-access-66zt7") pod "768126ea-6b59-47d9-a137-b71cd0695d33" (UID: "768126ea-6b59-47d9-a137-b71cd0695d33"). InnerVolumeSpecName "kube-api-access-66zt7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:17:15.347538 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.347507 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/768126ea-6b59-47d9-a137-b71cd0695d33-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "768126ea-6b59-47d9-a137-b71cd0695d33" (UID: "768126ea-6b59-47d9-a137-b71cd0695d33"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:17:15.347609 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.347546 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "768126ea-6b59-47d9-a137-b71cd0695d33" (UID: "768126ea-6b59-47d9-a137-b71cd0695d33"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:17:15.347609 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.347587 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/768126ea-6b59-47d9-a137-b71cd0695d33-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "768126ea-6b59-47d9-a137-b71cd0695d33" (UID: "768126ea-6b59-47d9-a137-b71cd0695d33"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:17:15.347685 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.347626 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "768126ea-6b59-47d9-a137-b71cd0695d33" (UID: "768126ea-6b59-47d9-a137-b71cd0695d33"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:17:15.356265 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.356236 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/768126ea-6b59-47d9-a137-b71cd0695d33-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "768126ea-6b59-47d9-a137-b71cd0695d33" (UID: "768126ea-6b59-47d9-a137-b71cd0695d33"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:17:15.446407 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.446377 2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/768126ea-6b59-47d9-a137-b71cd0695d33-trusted-ca\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:17:15.446407 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.446402 2556 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/768126ea-6b59-47d9-a137-b71cd0695d33-ca-trust-extracted\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:17:15.446407 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.446412 2556 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/768126ea-6b59-47d9-a137-b71cd0695d33-registry-certificates\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:17:15.446629 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.446420 2556 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-bound-sa-token\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:17:15.446629 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.446429 2556 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/768126ea-6b59-47d9-a137-b71cd0695d33-installation-pull-secrets\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:17:15.446629 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.446437 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66zt7\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-kube-api-access-66zt7\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:17:15.446629 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.446447 2556 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/768126ea-6b59-47d9-a137-b71cd0695d33-image-registry-private-configuration\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:17:15.446629 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:15.446455 2556 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/768126ea-6b59-47d9-a137-b71cd0695d33-registry-tls\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:17:16.114953 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:16.114914 2556 generic.go:356] "Generic (PLEG): container finished" podID="768126ea-6b59-47d9-a137-b71cd0695d33" containerID="29a3aefd384da667da1c7cab064015331e96dc9d581cc01c32b34c40d30a8e76" exitCode=0 Jul 02 19:17:16.115423 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:16.114974 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" event={"ID":"768126ea-6b59-47d9-a137-b71cd0695d33","Type":"ContainerDied","Data":"29a3aefd384da667da1c7cab064015331e96dc9d581cc01c32b34c40d30a8e76"} Jul 02 19:17:16.115423 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:16.114977 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" Jul 02 19:17:16.115423 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:16.115041 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cddd996cc-6qvxb" event={"ID":"768126ea-6b59-47d9-a137-b71cd0695d33","Type":"ContainerDied","Data":"aa455b050436e12646982b1f336925b99f0a50144705d79db4ede9e8a3d0e54c"} Jul 02 19:17:16.116831 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:16.116809 2556 scope.go:117] "RemoveContainer" containerID="29a3aefd384da667da1c7cab064015331e96dc9d581cc01c32b34c40d30a8e76" Jul 02 19:17:16.124379 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:16.124364 2556 scope.go:117] "RemoveContainer" containerID="29a3aefd384da667da1c7cab064015331e96dc9d581cc01c32b34c40d30a8e76" Jul 02 19:17:16.124639 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:17:16.124618 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29a3aefd384da667da1c7cab064015331e96dc9d581cc01c32b34c40d30a8e76\": container with ID starting with 29a3aefd384da667da1c7cab064015331e96dc9d581cc01c32b34c40d30a8e76 not found: ID does not exist" containerID="29a3aefd384da667da1c7cab064015331e96dc9d581cc01c32b34c40d30a8e76" Jul 02 19:17:16.124686 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:16.124647 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29a3aefd384da667da1c7cab064015331e96dc9d581cc01c32b34c40d30a8e76"} err="failed to get container status \"29a3aefd384da667da1c7cab064015331e96dc9d581cc01c32b34c40d30a8e76\": rpc error: code = NotFound desc = could not find container \"29a3aefd384da667da1c7cab064015331e96dc9d581cc01c32b34c40d30a8e76\": container with ID starting with 29a3aefd384da667da1c7cab064015331e96dc9d581cc01c32b34c40d30a8e76 not found: ID does not exist" Jul 02 19:17:16.135278 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:16.135257 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cddd996cc-6qvxb"] Jul 02 19:17:16.140427 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:16.140409 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6cddd996cc-6qvxb"] Jul 02 19:17:17.873633 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:17.873601 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8t8d8_171c244d-2dda-414a-a89f-c56e1a47c97c/serve-healthcheck-canary/0.log" Jul 02 19:17:17.978638 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:17.978604 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="768126ea-6b59-47d9-a137-b71cd0695d33" path="/var/lib/kubelet/pods/768126ea-6b59-47d9-a137-b71cd0695d33/volumes" Jul 02 19:17:41.852021 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:41.851973 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs\") pod \"network-metrics-daemon-pjsd8\" (UID: \"75464ead-f0fb-4ea8-beba-55b639087860\") " pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:17:41.854235 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:41.854216 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75464ead-f0fb-4ea8-beba-55b639087860-metrics-certs\") pod \"network-metrics-daemon-pjsd8\" (UID: \"75464ead-f0fb-4ea8-beba-55b639087860\") " pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:17:42.075772 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:42.075742 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-shdtl\"" Jul 02 19:17:42.084161 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:42.084123 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjsd8" Jul 02 19:17:42.211869 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:42.211838 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pjsd8"] Jul 02 19:17:42.214580 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:17:42.214554 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75464ead_f0fb_4ea8_beba_55b639087860.slice/crio-2a3796d48639ab4fa26401e387097e4139b8b78aa1ba9eebc289c43e2837cf36 WatchSource:0}: Error finding container 2a3796d48639ab4fa26401e387097e4139b8b78aa1ba9eebc289c43e2837cf36: Status 404 returned error can't find the container with id 2a3796d48639ab4fa26401e387097e4139b8b78aa1ba9eebc289c43e2837cf36 Jul 02 19:17:43.199804 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:43.199772 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pjsd8" event={"ID":"75464ead-f0fb-4ea8-beba-55b639087860","Type":"ContainerStarted","Data":"2a3796d48639ab4fa26401e387097e4139b8b78aa1ba9eebc289c43e2837cf36"} Jul 02 19:17:44.205404 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:44.205362 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pjsd8" event={"ID":"75464ead-f0fb-4ea8-beba-55b639087860","Type":"ContainerStarted","Data":"ff977af685c3a6d6afff5e83e7014a1446840941bbbd370d77bf57af63781c6f"} Jul 02 19:17:44.205783 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:44.205422 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pjsd8" event={"ID":"75464ead-f0fb-4ea8-beba-55b639087860","Type":"ContainerStarted","Data":"a7eef4e9e84c62cd37eaff1728726445fdceb8e072b138f3de657b11095c010d"} Jul 02 19:17:44.224935 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:17:44.224881 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pjsd8" podStartSLOduration=374.922458688 podStartE2EDuration="6m16.224866191s" totalImagesPullingTime="1.302407503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:17:44.223778189 +0000 UTC m=+376.770503196" watchObservedRunningTime="2026-07-02 19:17:44.224866191 +0000 UTC m=+376.771591197" Jul 02 19:18:13.893386 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:18:13.893336 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" podUID="535b962d-6d94-4e02-bd83-4411a9f5fd05" Jul 02 19:18:14.302059 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:14.302024 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:18:16.011470 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:16.011438 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-x6qcl\" (UID: \"535b962d-6d94-4e02-bd83-4411a9f5fd05\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:18:16.013971 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:16.013943 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/535b962d-6d94-4e02-bd83-4411a9f5fd05-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-x6qcl\" (UID: \"535b962d-6d94-4e02-bd83-4411a9f5fd05\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:18:16.105338 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:16.105307 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-86ffp\"" Jul 02 19:18:16.112860 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:16.112841 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" Jul 02 19:18:16.236616 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:16.236553 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl"] Jul 02 19:18:16.239113 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:18:16.239086 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod535b962d_6d94_4e02_bd83_4411a9f5fd05.slice/crio-2abda15e7feff2ebfc704ce6d2d0c33296d9865548d51d932e317278ea303296 WatchSource:0}: Error finding container 2abda15e7feff2ebfc704ce6d2d0c33296d9865548d51d932e317278ea303296: Status 404 returned error can't find the container with id 2abda15e7feff2ebfc704ce6d2d0c33296d9865548d51d932e317278ea303296 Jul 02 19:18:16.309094 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:16.309006 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" event={"ID":"535b962d-6d94-4e02-bd83-4411a9f5fd05","Type":"ContainerStarted","Data":"2abda15e7feff2ebfc704ce6d2d0c33296d9865548d51d932e317278ea303296"} Jul 02 19:18:18.319376 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:18.319335 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" event={"ID":"535b962d-6d94-4e02-bd83-4411a9f5fd05","Type":"ContainerStarted","Data":"870de71aac8b71bd741e7bad6fd39735e6ce41c1990c072f68ae0e9c9c9c0f5b"} Jul 02 19:18:18.335019 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:18.334971 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-x6qcl" podStartSLOduration=395.150483832 podStartE2EDuration="6m36.334958455s" totalImagesPullingTime="1.184474623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:11:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:18:18.334474753 +0000 UTC m=+410.881199760" watchObservedRunningTime="2026-07-02 19:18:18.334958455 +0000 UTC m=+410.881683461" Jul 02 19:18:19.656774 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:19.656739 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6757d494d-7dqvd"] Jul 02 19:18:44.679951 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:44.679884 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6757d494d-7dqvd" podUID="0efd7e51-17b3-406e-85e0-bc1237d2452b" containerName="console" containerID="cri-o://85afa0eb246ec718c6c43dccdbeeaf34431aa418aecb44e65c220c1d30ac7795" gracePeriod=15 Jul 02 19:18:44.919060 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:44.919024 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:18:45.003503 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.003401 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-trusted-ca-bundle\") pod \"0efd7e51-17b3-406e-85e0-bc1237d2452b\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " Jul 02 19:18:45.003503 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.003461 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-service-ca\") pod \"0efd7e51-17b3-406e-85e0-bc1237d2452b\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " Jul 02 19:18:45.003503 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.003483 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98dsl\" (UniqueName: \"kubernetes.io/projected/0efd7e51-17b3-406e-85e0-bc1237d2452b-kube-api-access-98dsl\") pod \"0efd7e51-17b3-406e-85e0-bc1237d2452b\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " Jul 02 19:18:45.003794 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.003509 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0efd7e51-17b3-406e-85e0-bc1237d2452b-console-oauth-config\") pod \"0efd7e51-17b3-406e-85e0-bc1237d2452b\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " Jul 02 19:18:45.003794 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.003525 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0efd7e51-17b3-406e-85e0-bc1237d2452b-console-serving-cert\") pod \"0efd7e51-17b3-406e-85e0-bc1237d2452b\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " Jul 02 19:18:45.003794 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.003568 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-console-config\") pod \"0efd7e51-17b3-406e-85e0-bc1237d2452b\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " Jul 02 19:18:45.003794 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.003593 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-oauth-serving-cert\") pod \"0efd7e51-17b3-406e-85e0-bc1237d2452b\" (UID: \"0efd7e51-17b3-406e-85e0-bc1237d2452b\") " Jul 02 19:18:45.003991 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.003837 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0efd7e51-17b3-406e-85e0-bc1237d2452b" (UID: "0efd7e51-17b3-406e-85e0-bc1237d2452b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:18:45.004048 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.004023 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-console-config" (OuterVolumeSpecName: "console-config") pod "0efd7e51-17b3-406e-85e0-bc1237d2452b" (UID: "0efd7e51-17b3-406e-85e0-bc1237d2452b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:18:45.004163 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.004111 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0efd7e51-17b3-406e-85e0-bc1237d2452b" (UID: "0efd7e51-17b3-406e-85e0-bc1237d2452b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:18:45.004271 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.004206 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-service-ca" (OuterVolumeSpecName: "service-ca") pod "0efd7e51-17b3-406e-85e0-bc1237d2452b" (UID: "0efd7e51-17b3-406e-85e0-bc1237d2452b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:18:45.005887 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.005863 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0efd7e51-17b3-406e-85e0-bc1237d2452b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0efd7e51-17b3-406e-85e0-bc1237d2452b" (UID: "0efd7e51-17b3-406e-85e0-bc1237d2452b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:18:45.005953 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.005901 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0efd7e51-17b3-406e-85e0-bc1237d2452b-kube-api-access-98dsl" (OuterVolumeSpecName: "kube-api-access-98dsl") pod "0efd7e51-17b3-406e-85e0-bc1237d2452b" (UID: "0efd7e51-17b3-406e-85e0-bc1237d2452b"). InnerVolumeSpecName "kube-api-access-98dsl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:18:45.005989 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.005950 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0efd7e51-17b3-406e-85e0-bc1237d2452b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0efd7e51-17b3-406e-85e0-bc1237d2452b" (UID: "0efd7e51-17b3-406e-85e0-bc1237d2452b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:18:45.104498 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.104445 2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-console-config\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:18:45.104498 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.104491 2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-oauth-serving-cert\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:18:45.104498 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.104502 2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-trusted-ca-bundle\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:18:45.104498 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.104510 2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0efd7e51-17b3-406e-85e0-bc1237d2452b-service-ca\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:18:45.104748 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.104519 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98dsl\" (UniqueName: \"kubernetes.io/projected/0efd7e51-17b3-406e-85e0-bc1237d2452b-kube-api-access-98dsl\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:18:45.104748 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.104529 2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0efd7e51-17b3-406e-85e0-bc1237d2452b-console-oauth-config\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:18:45.104748 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.104537 2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0efd7e51-17b3-406e-85e0-bc1237d2452b-console-serving-cert\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:18:45.409243 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.409168 2556 generic.go:356] "Generic (PLEG): container finished" podID="0efd7e51-17b3-406e-85e0-bc1237d2452b" containerID="85afa0eb246ec718c6c43dccdbeeaf34431aa418aecb44e65c220c1d30ac7795" exitCode=0 Jul 02 19:18:45.409243 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.409207 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6757d494d-7dqvd" Jul 02 19:18:45.409243 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.409228 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6757d494d-7dqvd" event={"ID":"0efd7e51-17b3-406e-85e0-bc1237d2452b","Type":"ContainerDied","Data":"85afa0eb246ec718c6c43dccdbeeaf34431aa418aecb44e65c220c1d30ac7795"} Jul 02 19:18:45.409459 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.409267 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6757d494d-7dqvd" event={"ID":"0efd7e51-17b3-406e-85e0-bc1237d2452b","Type":"ContainerDied","Data":"0659de95b4404d5b42b82586b71513cad4d7dbc96fd1934442cd5dd3e8ab098e"} Jul 02 19:18:45.409459 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.409288 2556 scope.go:117] "RemoveContainer" containerID="85afa0eb246ec718c6c43dccdbeeaf34431aa418aecb44e65c220c1d30ac7795" Jul 02 19:18:45.417185 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.417048 2556 scope.go:117] "RemoveContainer" containerID="85afa0eb246ec718c6c43dccdbeeaf34431aa418aecb44e65c220c1d30ac7795" Jul 02 19:18:45.417429 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:18:45.417400 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85afa0eb246ec718c6c43dccdbeeaf34431aa418aecb44e65c220c1d30ac7795\": container with ID starting with 85afa0eb246ec718c6c43dccdbeeaf34431aa418aecb44e65c220c1d30ac7795 not found: ID does not exist" containerID="85afa0eb246ec718c6c43dccdbeeaf34431aa418aecb44e65c220c1d30ac7795" Jul 02 19:18:45.417490 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.417434 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85afa0eb246ec718c6c43dccdbeeaf34431aa418aecb44e65c220c1d30ac7795"} err="failed to get container status \"85afa0eb246ec718c6c43dccdbeeaf34431aa418aecb44e65c220c1d30ac7795\": rpc error: code = NotFound desc = could not find container \"85afa0eb246ec718c6c43dccdbeeaf34431aa418aecb44e65c220c1d30ac7795\": container with ID starting with 85afa0eb246ec718c6c43dccdbeeaf34431aa418aecb44e65c220c1d30ac7795 not found: ID does not exist" Jul 02 19:18:45.429853 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.429829 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6757d494d-7dqvd"] Jul 02 19:18:45.435445 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.435422 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6757d494d-7dqvd"] Jul 02 19:18:45.977563 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:18:45.977532 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0efd7e51-17b3-406e-85e0-bc1237d2452b" path="/var/lib/kubelet/pods/0efd7e51-17b3-406e-85e0-bc1237d2452b/volumes" Jul 02 19:19:21.636957 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.636915 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5554d9cbbb-nqlrl"] Jul 02 19:19:21.637542 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.637250 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="768126ea-6b59-47d9-a137-b71cd0695d33" containerName="registry" Jul 02 19:19:21.637542 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.637271 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="768126ea-6b59-47d9-a137-b71cd0695d33" containerName="registry" Jul 02 19:19:21.637542 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.637293 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0efd7e51-17b3-406e-85e0-bc1237d2452b" containerName="console" Jul 02 19:19:21.637542 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.637302 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0efd7e51-17b3-406e-85e0-bc1237d2452b" containerName="console" Jul 02 19:19:21.637542 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.637363 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0efd7e51-17b3-406e-85e0-bc1237d2452b" containerName="console" Jul 02 19:19:21.637542 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.637377 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="768126ea-6b59-47d9-a137-b71cd0695d33" containerName="registry" Jul 02 19:19:21.640184 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.640138 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.642734 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.642706 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 02 19:19:21.642859 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.642787 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 02 19:19:21.642859 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.642848 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 02 19:19:21.645215 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.645197 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 02 19:19:21.645340 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.645293 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-m6c26\"" Jul 02 19:19:21.645410 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.645339 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 02 19:19:21.645473 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.645410 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 02 19:19:21.645522 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.645474 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 02 19:19:21.649273 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.649249 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 02 19:19:21.650669 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.650646 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5554d9cbbb-nqlrl"] Jul 02 19:19:21.756483 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.756453 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzlwv\" (UniqueName: \"kubernetes.io/projected/dc61eeea-b62b-40a8-93e0-5bb748b7c964-kube-api-access-lzlwv\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.756483 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.756487 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-console-config\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.756696 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.756512 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-trusted-ca-bundle\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.756696 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.756542 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-oauth-serving-cert\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.756696 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.756560 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc61eeea-b62b-40a8-93e0-5bb748b7c964-console-oauth-config\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.756696 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.756577 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc61eeea-b62b-40a8-93e0-5bb748b7c964-console-serving-cert\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.756696 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.756624 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-service-ca\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.857882 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.857839 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-trusted-ca-bundle\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.857995 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.857898 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-oauth-serving-cert\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.858033 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.858009 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc61eeea-b62b-40a8-93e0-5bb748b7c964-console-oauth-config\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.858067 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.858051 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc61eeea-b62b-40a8-93e0-5bb748b7c964-console-serving-cert\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.858108 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.858093 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-service-ca\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.858168 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.858130 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzlwv\" (UniqueName: \"kubernetes.io/projected/dc61eeea-b62b-40a8-93e0-5bb748b7c964-kube-api-access-lzlwv\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.858219 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.858176 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-console-config\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.858666 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.858641 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-oauth-serving-cert\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.858813 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.858790 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-console-config\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.858917 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.858897 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-service-ca\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.859026 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.859010 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-trusted-ca-bundle\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.860466 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.860446 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc61eeea-b62b-40a8-93e0-5bb748b7c964-console-oauth-config\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.860556 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.860534 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc61eeea-b62b-40a8-93e0-5bb748b7c964-console-serving-cert\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.866758 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.866741 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzlwv\" (UniqueName: \"kubernetes.io/projected/dc61eeea-b62b-40a8-93e0-5bb748b7c964-kube-api-access-lzlwv\") pod \"console-5554d9cbbb-nqlrl\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:21.950544 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:21.950514 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:22.075193 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:22.075163 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5554d9cbbb-nqlrl"] Jul 02 19:19:22.078477 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:19:22.078454 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc61eeea_b62b_40a8_93e0_5bb748b7c964.slice/crio-3be1e922581f706d5a3fac262044e5fe23b95a1669c3a4b3c39d8a2667b99737 WatchSource:0}: Error finding container 3be1e922581f706d5a3fac262044e5fe23b95a1669c3a4b3c39d8a2667b99737: Status 404 returned error can't find the container with id 3be1e922581f706d5a3fac262044e5fe23b95a1669c3a4b3c39d8a2667b99737 Jul 02 19:19:22.533255 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:22.533219 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5554d9cbbb-nqlrl" event={"ID":"dc61eeea-b62b-40a8-93e0-5bb748b7c964","Type":"ContainerStarted","Data":"48d85ebcf102ad3c62eac9da675bb0c4e21a5a2509be0c40e4176f53dd94301f"} Jul 02 19:19:22.533432 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:22.533267 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5554d9cbbb-nqlrl" event={"ID":"dc61eeea-b62b-40a8-93e0-5bb748b7c964","Type":"ContainerStarted","Data":"3be1e922581f706d5a3fac262044e5fe23b95a1669c3a4b3c39d8a2667b99737"} Jul 02 19:19:22.551231 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:22.551181 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5554d9cbbb-nqlrl" podStartSLOduration=1.5511679200000001 podStartE2EDuration="1.55116792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:19:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:19:22.549505804 +0000 UTC m=+475.096230828" watchObservedRunningTime="2026-07-02 19:19:22.55116792 +0000 UTC m=+475.097892928" Jul 02 19:19:31.950723 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:31.950684 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:31.951226 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:31.950782 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:31.955337 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:31.955316 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:19:32.575545 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:19:32.575512 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:20:46.794631 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:46.794596 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v"] Jul 02 19:20:46.797740 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:46.797723 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v" Jul 02 19:20:46.800502 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:46.800477 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 19:20:46.801475 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:46.801460 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 19:20:46.801578 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:46.801517 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bkmn9\"" Jul 02 19:20:46.805576 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:46.805554 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v"] Jul 02 19:20:46.823605 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:46.823576 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lq7c\" (UniqueName: \"kubernetes.io/projected/dcea4d83-72c3-4f54-ab52-6bed2b781a8a-kube-api-access-2lq7c\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v\" (UID: \"dcea4d83-72c3-4f54-ab52-6bed2b781a8a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v" Jul 02 19:20:46.823725 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:46.823617 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcea4d83-72c3-4f54-ab52-6bed2b781a8a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v\" (UID: \"dcea4d83-72c3-4f54-ab52-6bed2b781a8a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v" Jul 02 19:20:46.823725 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:46.823664 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcea4d83-72c3-4f54-ab52-6bed2b781a8a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v\" (UID: \"dcea4d83-72c3-4f54-ab52-6bed2b781a8a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v" Jul 02 19:20:46.924354 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:46.924311 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcea4d83-72c3-4f54-ab52-6bed2b781a8a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v\" (UID: \"dcea4d83-72c3-4f54-ab52-6bed2b781a8a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v" Jul 02 19:20:46.924549 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:46.924371 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lq7c\" (UniqueName: \"kubernetes.io/projected/dcea4d83-72c3-4f54-ab52-6bed2b781a8a-kube-api-access-2lq7c\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v\" (UID: \"dcea4d83-72c3-4f54-ab52-6bed2b781a8a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v" Jul 02 19:20:46.924549 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:46.924412 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcea4d83-72c3-4f54-ab52-6bed2b781a8a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v\" (UID: \"dcea4d83-72c3-4f54-ab52-6bed2b781a8a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v" Jul 02 19:20:46.924682 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:46.924662 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcea4d83-72c3-4f54-ab52-6bed2b781a8a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v\" (UID: \"dcea4d83-72c3-4f54-ab52-6bed2b781a8a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v" Jul 02 19:20:46.924735 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:46.924696 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcea4d83-72c3-4f54-ab52-6bed2b781a8a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v\" (UID: \"dcea4d83-72c3-4f54-ab52-6bed2b781a8a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v" Jul 02 19:20:46.932381 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:46.932353 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lq7c\" (UniqueName: \"kubernetes.io/projected/dcea4d83-72c3-4f54-ab52-6bed2b781a8a-kube-api-access-2lq7c\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v\" (UID: \"dcea4d83-72c3-4f54-ab52-6bed2b781a8a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v" Jul 02 19:20:47.106550 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:47.106454 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v" Jul 02 19:20:47.234308 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:47.234268 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v"] Jul 02 19:20:47.237197 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:20:47.237169 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcea4d83_72c3_4f54_ab52_6bed2b781a8a.slice/crio-386a365062303d0bd98eb35c85a926f997adedac8f55d58852fb3ae8592a96f7 WatchSource:0}: Error finding container 386a365062303d0bd98eb35c85a926f997adedac8f55d58852fb3ae8592a96f7: Status 404 returned error can't find the container with id 386a365062303d0bd98eb35c85a926f997adedac8f55d58852fb3ae8592a96f7 Jul 02 19:20:47.815933 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:47.815900 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v" event={"ID":"dcea4d83-72c3-4f54-ab52-6bed2b781a8a","Type":"ContainerStarted","Data":"386a365062303d0bd98eb35c85a926f997adedac8f55d58852fb3ae8592a96f7"} Jul 02 19:20:56.847053 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:56.847014 2556 generic.go:356] "Generic (PLEG): container finished" podID="dcea4d83-72c3-4f54-ab52-6bed2b781a8a" containerID="7fa73018894607f43fcbaf00ea37abfda855307a0c57f78e41ef51188d141337" exitCode=0 Jul 02 19:20:56.847471 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:56.847086 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v" event={"ID":"dcea4d83-72c3-4f54-ab52-6bed2b781a8a","Type":"ContainerDied","Data":"7fa73018894607f43fcbaf00ea37abfda855307a0c57f78e41ef51188d141337"} Jul 02 19:20:58.855843 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:58.855808 2556 generic.go:356] "Generic (PLEG): container finished" podID="dcea4d83-72c3-4f54-ab52-6bed2b781a8a" containerID="ccae98fa761392d8c1a14a2400a033d11ce6591f14246896aff3d2ebf8151d1b" exitCode=0 Jul 02 19:20:58.856280 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:20:58.855878 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v" event={"ID":"dcea4d83-72c3-4f54-ab52-6bed2b781a8a","Type":"ContainerDied","Data":"ccae98fa761392d8c1a14a2400a033d11ce6591f14246896aff3d2ebf8151d1b"} Jul 02 19:21:07.889503 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:07.889469 2556 generic.go:356] "Generic (PLEG): container finished" podID="dcea4d83-72c3-4f54-ab52-6bed2b781a8a" containerID="804b5c2162f3f5760a889e5db8ae9d644fd1893e779eee44f48881df41fc0e20" exitCode=0 Jul 02 19:21:07.889919 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:07.889526 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v" event={"ID":"dcea4d83-72c3-4f54-ab52-6bed2b781a8a","Type":"ContainerDied","Data":"804b5c2162f3f5760a889e5db8ae9d644fd1893e779eee44f48881df41fc0e20"} Jul 02 19:21:09.050280 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:09.050247 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v" Jul 02 19:21:09.093490 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:09.093457 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcea4d83-72c3-4f54-ab52-6bed2b781a8a-bundle\") pod \"dcea4d83-72c3-4f54-ab52-6bed2b781a8a\" (UID: \"dcea4d83-72c3-4f54-ab52-6bed2b781a8a\") " Jul 02 19:21:09.093653 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:09.093518 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lq7c\" (UniqueName: \"kubernetes.io/projected/dcea4d83-72c3-4f54-ab52-6bed2b781a8a-kube-api-access-2lq7c\") pod \"dcea4d83-72c3-4f54-ab52-6bed2b781a8a\" (UID: \"dcea4d83-72c3-4f54-ab52-6bed2b781a8a\") " Jul 02 19:21:09.093653 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:09.093558 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcea4d83-72c3-4f54-ab52-6bed2b781a8a-util\") pod \"dcea4d83-72c3-4f54-ab52-6bed2b781a8a\" (UID: \"dcea4d83-72c3-4f54-ab52-6bed2b781a8a\") " Jul 02 19:21:09.094266 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:09.094233 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcea4d83-72c3-4f54-ab52-6bed2b781a8a-bundle" (OuterVolumeSpecName: "bundle") pod "dcea4d83-72c3-4f54-ab52-6bed2b781a8a" (UID: "dcea4d83-72c3-4f54-ab52-6bed2b781a8a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:21:09.095746 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:09.095722 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcea4d83-72c3-4f54-ab52-6bed2b781a8a-kube-api-access-2lq7c" (OuterVolumeSpecName: "kube-api-access-2lq7c") pod "dcea4d83-72c3-4f54-ab52-6bed2b781a8a" (UID: "dcea4d83-72c3-4f54-ab52-6bed2b781a8a"). InnerVolumeSpecName "kube-api-access-2lq7c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:21:09.097689 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:09.097668 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcea4d83-72c3-4f54-ab52-6bed2b781a8a-util" (OuterVolumeSpecName: "util") pod "dcea4d83-72c3-4f54-ab52-6bed2b781a8a" (UID: "dcea4d83-72c3-4f54-ab52-6bed2b781a8a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:21:09.194546 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:09.194502 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lq7c\" (UniqueName: \"kubernetes.io/projected/dcea4d83-72c3-4f54-ab52-6bed2b781a8a-kube-api-access-2lq7c\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:21:09.194546 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:09.194540 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcea4d83-72c3-4f54-ab52-6bed2b781a8a-util\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:21:09.194546 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:09.194549 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcea4d83-72c3-4f54-ab52-6bed2b781a8a-bundle\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:21:09.898212 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:09.898170 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v" event={"ID":"dcea4d83-72c3-4f54-ab52-6bed2b781a8a","Type":"ContainerDied","Data":"386a365062303d0bd98eb35c85a926f997adedac8f55d58852fb3ae8592a96f7"} Jul 02 19:21:09.898212 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:09.898215 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409kp5v" Jul 02 19:21:09.898411 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:09.898224 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="386a365062303d0bd98eb35c85a926f997adedac8f55d58852fb3ae8592a96f7" Jul 02 19:21:14.573776 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.573739 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zn5x2"] Jul 02 19:21:14.574278 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.573959 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcea4d83-72c3-4f54-ab52-6bed2b781a8a" containerName="util" Jul 02 19:21:14.574278 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.573970 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcea4d83-72c3-4f54-ab52-6bed2b781a8a" containerName="util" Jul 02 19:21:14.574278 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.573984 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcea4d83-72c3-4f54-ab52-6bed2b781a8a" containerName="extract" Jul 02 19:21:14.574278 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.573989 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcea4d83-72c3-4f54-ab52-6bed2b781a8a" containerName="extract" Jul 02 19:21:14.574278 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.573997 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcea4d83-72c3-4f54-ab52-6bed2b781a8a" containerName="pull" Jul 02 19:21:14.574278 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.574002 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcea4d83-72c3-4f54-ab52-6bed2b781a8a" containerName="pull" Jul 02 19:21:14.574278 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.574042 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcea4d83-72c3-4f54-ab52-6bed2b781a8a" containerName="extract" Jul 02 19:21:14.576127 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.576110 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zn5x2" Jul 02 19:21:14.578822 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.578797 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-f5dfz\"" Jul 02 19:21:14.578822 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.578818 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 02 19:21:14.578986 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.578797 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 02 19:21:14.593802 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.593779 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zn5x2"] Jul 02 19:21:14.629275 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.629243 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd0c04b3-20fb-45e6-9809-7e5902bf3b72-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zn5x2\" (UID: \"fd0c04b3-20fb-45e6-9809-7e5902bf3b72\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zn5x2" Jul 02 19:21:14.629413 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.629292 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kdpz\" (UniqueName: \"kubernetes.io/projected/fd0c04b3-20fb-45e6-9809-7e5902bf3b72-kube-api-access-2kdpz\") pod \"cert-manager-operator-controller-manager-55c444bf86-zn5x2\" (UID: \"fd0c04b3-20fb-45e6-9809-7e5902bf3b72\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zn5x2" Jul 02 19:21:14.730629 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.730587 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kdpz\" (UniqueName: \"kubernetes.io/projected/fd0c04b3-20fb-45e6-9809-7e5902bf3b72-kube-api-access-2kdpz\") pod \"cert-manager-operator-controller-manager-55c444bf86-zn5x2\" (UID: \"fd0c04b3-20fb-45e6-9809-7e5902bf3b72\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zn5x2" Jul 02 19:21:14.730814 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.730659 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd0c04b3-20fb-45e6-9809-7e5902bf3b72-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zn5x2\" (UID: \"fd0c04b3-20fb-45e6-9809-7e5902bf3b72\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zn5x2" Jul 02 19:21:14.731045 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.731023 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd0c04b3-20fb-45e6-9809-7e5902bf3b72-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zn5x2\" (UID: \"fd0c04b3-20fb-45e6-9809-7e5902bf3b72\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zn5x2" Jul 02 19:21:14.739672 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.739649 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kdpz\" (UniqueName: \"kubernetes.io/projected/fd0c04b3-20fb-45e6-9809-7e5902bf3b72-kube-api-access-2kdpz\") pod \"cert-manager-operator-controller-manager-55c444bf86-zn5x2\" (UID: \"fd0c04b3-20fb-45e6-9809-7e5902bf3b72\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zn5x2" Jul 02 19:21:14.885081 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:14.884987 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zn5x2" Jul 02 19:21:15.012891 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:15.012862 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zn5x2"] Jul 02 19:21:15.015778 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:21:15.015757 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd0c04b3_20fb_45e6_9809_7e5902bf3b72.slice/crio-6dfd306fbdb0c651d2ad0ee3b76d8d922a61afdd7df36aca60a401360d1b5327 WatchSource:0}: Error finding container 6dfd306fbdb0c651d2ad0ee3b76d8d922a61afdd7df36aca60a401360d1b5327: Status 404 returned error can't find the container with id 6dfd306fbdb0c651d2ad0ee3b76d8d922a61afdd7df36aca60a401360d1b5327 Jul 02 19:21:15.922983 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:15.922940 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zn5x2" event={"ID":"fd0c04b3-20fb-45e6-9809-7e5902bf3b72","Type":"ContainerStarted","Data":"6dfd306fbdb0c651d2ad0ee3b76d8d922a61afdd7df36aca60a401360d1b5327"} Jul 02 19:21:16.929289 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:16.929258 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zn5x2" event={"ID":"fd0c04b3-20fb-45e6-9809-7e5902bf3b72","Type":"ContainerStarted","Data":"27fc65382e38ea7a64bccdf9ec7cec69c642c9b4845226a3908001e944afbd28"} Jul 02 19:21:16.951132 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:16.951060 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zn5x2" podStartSLOduration=1.322517696 podStartE2EDuration="2.951040454s" totalImagesPullingTime="1.628522758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:21:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:21:16.950303336 +0000 UTC m=+589.497028353" watchObservedRunningTime="2026-07-02 19:21:16.951040454 +0000 UTC m=+589.497765465" Jul 02 19:21:18.248942 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.248900 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95"] Jul 02 19:21:18.251221 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.251198 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95" Jul 02 19:21:18.253986 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.253962 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 19:21:18.254102 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.254007 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bkmn9\"" Jul 02 19:21:18.254984 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.254946 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 19:21:18.262249 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.262230 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95"] Jul 02 19:21:18.356126 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.356088 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77a02f88-756d-4080-b5bc-eba068e9a01a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95\" (UID: \"77a02f88-756d-4080-b5bc-eba068e9a01a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95" Jul 02 19:21:18.356126 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.356128 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77a02f88-756d-4080-b5bc-eba068e9a01a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95\" (UID: \"77a02f88-756d-4080-b5bc-eba068e9a01a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95" Jul 02 19:21:18.356338 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.356221 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbkg2\" (UniqueName: \"kubernetes.io/projected/77a02f88-756d-4080-b5bc-eba068e9a01a-kube-api-access-gbkg2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95\" (UID: \"77a02f88-756d-4080-b5bc-eba068e9a01a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95" Jul 02 19:21:18.457072 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.457036 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbkg2\" (UniqueName: \"kubernetes.io/projected/77a02f88-756d-4080-b5bc-eba068e9a01a-kube-api-access-gbkg2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95\" (UID: \"77a02f88-756d-4080-b5bc-eba068e9a01a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95" Jul 02 19:21:18.457272 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.457163 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77a02f88-756d-4080-b5bc-eba068e9a01a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95\" (UID: \"77a02f88-756d-4080-b5bc-eba068e9a01a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95" Jul 02 19:21:18.457272 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.457193 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77a02f88-756d-4080-b5bc-eba068e9a01a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95\" (UID: \"77a02f88-756d-4080-b5bc-eba068e9a01a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95" Jul 02 19:21:18.457541 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.457521 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77a02f88-756d-4080-b5bc-eba068e9a01a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95\" (UID: \"77a02f88-756d-4080-b5bc-eba068e9a01a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95" Jul 02 19:21:18.457587 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.457561 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77a02f88-756d-4080-b5bc-eba068e9a01a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95\" (UID: \"77a02f88-756d-4080-b5bc-eba068e9a01a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95" Jul 02 19:21:18.471807 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.471781 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbkg2\" (UniqueName: \"kubernetes.io/projected/77a02f88-756d-4080-b5bc-eba068e9a01a-kube-api-access-gbkg2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95\" (UID: \"77a02f88-756d-4080-b5bc-eba068e9a01a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95" Jul 02 19:21:18.560186 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.560075 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95" Jul 02 19:21:18.689496 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.689360 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95"] Jul 02 19:21:18.692226 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:21:18.692185 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77a02f88_756d_4080_b5bc_eba068e9a01a.slice/crio-aa8c892743f963db58d681ab1be0a505c53e5133f4de306e5a2afdbe38edc7ae WatchSource:0}: Error finding container aa8c892743f963db58d681ab1be0a505c53e5133f4de306e5a2afdbe38edc7ae: Status 404 returned error can't find the container with id aa8c892743f963db58d681ab1be0a505c53e5133f4de306e5a2afdbe38edc7ae Jul 02 19:21:18.937844 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.937814 2556 generic.go:356] "Generic (PLEG): container finished" podID="77a02f88-756d-4080-b5bc-eba068e9a01a" containerID="40626882c70a1d1901c878d2ea145418b6766dc78a3acd873228fc8bab7d53d6" exitCode=0 Jul 02 19:21:18.938017 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.937890 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95" event={"ID":"77a02f88-756d-4080-b5bc-eba068e9a01a","Type":"ContainerDied","Data":"40626882c70a1d1901c878d2ea145418b6766dc78a3acd873228fc8bab7d53d6"} Jul 02 19:21:18.938017 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:18.937928 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95" event={"ID":"77a02f88-756d-4080-b5bc-eba068e9a01a","Type":"ContainerStarted","Data":"aa8c892743f963db58d681ab1be0a505c53e5133f4de306e5a2afdbe38edc7ae"} Jul 02 19:21:20.677635 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:20.677598 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5zpnd"] Jul 02 19:21:20.683758 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:20.683737 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5zpnd" Jul 02 19:21:20.686534 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:20.686509 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 02 19:21:20.686645 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:20.686509 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-28bc6\"" Jul 02 19:21:20.687532 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:20.687511 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 02 19:21:20.697942 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:20.697916 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5zpnd"] Jul 02 19:21:20.775500 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:20.775462 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v295h\" (UniqueName: \"kubernetes.io/projected/644ac5a6-9bc9-4226-a55a-1a1b1e4505a8-kube-api-access-v295h\") pod \"cert-manager-cainjector-559cdd8c87-5zpnd\" (UID: \"644ac5a6-9bc9-4226-a55a-1a1b1e4505a8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5zpnd" Jul 02 19:21:20.775681 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:20.775533 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/644ac5a6-9bc9-4226-a55a-1a1b1e4505a8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5zpnd\" (UID: \"644ac5a6-9bc9-4226-a55a-1a1b1e4505a8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5zpnd" Jul 02 19:21:20.876719 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:20.876674 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v295h\" (UniqueName: \"kubernetes.io/projected/644ac5a6-9bc9-4226-a55a-1a1b1e4505a8-kube-api-access-v295h\") pod \"cert-manager-cainjector-559cdd8c87-5zpnd\" (UID: \"644ac5a6-9bc9-4226-a55a-1a1b1e4505a8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5zpnd" Jul 02 19:21:20.876893 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:20.876770 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/644ac5a6-9bc9-4226-a55a-1a1b1e4505a8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5zpnd\" (UID: \"644ac5a6-9bc9-4226-a55a-1a1b1e4505a8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5zpnd" Jul 02 19:21:20.885554 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:20.885515 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/644ac5a6-9bc9-4226-a55a-1a1b1e4505a8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5zpnd\" (UID: \"644ac5a6-9bc9-4226-a55a-1a1b1e4505a8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5zpnd" Jul 02 19:21:20.885708 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:20.885632 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v295h\" (UniqueName: \"kubernetes.io/projected/644ac5a6-9bc9-4226-a55a-1a1b1e4505a8-kube-api-access-v295h\") pod \"cert-manager-cainjector-559cdd8c87-5zpnd\" (UID: \"644ac5a6-9bc9-4226-a55a-1a1b1e4505a8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5zpnd" Jul 02 19:21:20.994025 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:20.994008 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5zpnd" Jul 02 19:21:21.128068 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:21.128034 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5zpnd"] Jul 02 19:21:21.131052 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:21:21.131028 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod644ac5a6_9bc9_4226_a55a_1a1b1e4505a8.slice/crio-f536cff0c0b92790a5f38ebc677afd209358ae5bf47f2e826319d7bea4bd4f1e WatchSource:0}: Error finding container f536cff0c0b92790a5f38ebc677afd209358ae5bf47f2e826319d7bea4bd4f1e: Status 404 returned error can't find the container with id f536cff0c0b92790a5f38ebc677afd209358ae5bf47f2e826319d7bea4bd4f1e Jul 02 19:21:21.951918 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:21.951880 2556 generic.go:356] "Generic (PLEG): container finished" podID="77a02f88-756d-4080-b5bc-eba068e9a01a" containerID="69cd4b6d595acec3c2270ac6dfc9ff3098a521b3b90b7336f10296cd8ed7f6a8" exitCode=0 Jul 02 19:21:21.952368 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:21.951959 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95" event={"ID":"77a02f88-756d-4080-b5bc-eba068e9a01a","Type":"ContainerDied","Data":"69cd4b6d595acec3c2270ac6dfc9ff3098a521b3b90b7336f10296cd8ed7f6a8"} Jul 02 19:21:21.953187 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:21.953166 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5zpnd" event={"ID":"644ac5a6-9bc9-4226-a55a-1a1b1e4505a8","Type":"ContainerStarted","Data":"f536cff0c0b92790a5f38ebc677afd209358ae5bf47f2e826319d7bea4bd4f1e"} Jul 02 19:21:22.958929 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:22.958892 2556 generic.go:356] "Generic (PLEG): container finished" podID="77a02f88-756d-4080-b5bc-eba068e9a01a" containerID="0b29a0ace72aa6ebacc889eb69da659aeb954828d3baed45319625651079f94c" exitCode=0 Jul 02 19:21:22.959317 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:22.958954 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95" event={"ID":"77a02f88-756d-4080-b5bc-eba068e9a01a","Type":"ContainerDied","Data":"0b29a0ace72aa6ebacc889eb69da659aeb954828d3baed45319625651079f94c"} Jul 02 19:21:24.111895 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:24.111857 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95" Jul 02 19:21:24.201797 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:24.201756 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77a02f88-756d-4080-b5bc-eba068e9a01a-bundle\") pod \"77a02f88-756d-4080-b5bc-eba068e9a01a\" (UID: \"77a02f88-756d-4080-b5bc-eba068e9a01a\") " Jul 02 19:21:24.201797 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:24.201803 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77a02f88-756d-4080-b5bc-eba068e9a01a-util\") pod \"77a02f88-756d-4080-b5bc-eba068e9a01a\" (UID: \"77a02f88-756d-4080-b5bc-eba068e9a01a\") " Jul 02 19:21:24.202008 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:24.201861 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbkg2\" (UniqueName: \"kubernetes.io/projected/77a02f88-756d-4080-b5bc-eba068e9a01a-kube-api-access-gbkg2\") pod \"77a02f88-756d-4080-b5bc-eba068e9a01a\" (UID: \"77a02f88-756d-4080-b5bc-eba068e9a01a\") " Jul 02 19:21:24.202297 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:24.202250 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77a02f88-756d-4080-b5bc-eba068e9a01a-bundle" (OuterVolumeSpecName: "bundle") pod "77a02f88-756d-4080-b5bc-eba068e9a01a" (UID: "77a02f88-756d-4080-b5bc-eba068e9a01a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:21:24.204510 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:24.204479 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77a02f88-756d-4080-b5bc-eba068e9a01a-kube-api-access-gbkg2" (OuterVolumeSpecName: "kube-api-access-gbkg2") pod "77a02f88-756d-4080-b5bc-eba068e9a01a" (UID: "77a02f88-756d-4080-b5bc-eba068e9a01a"). InnerVolumeSpecName "kube-api-access-gbkg2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:21:24.208004 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:24.207974 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77a02f88-756d-4080-b5bc-eba068e9a01a-util" (OuterVolumeSpecName: "util") pod "77a02f88-756d-4080-b5bc-eba068e9a01a" (UID: "77a02f88-756d-4080-b5bc-eba068e9a01a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:21:24.303173 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:24.303083 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77a02f88-756d-4080-b5bc-eba068e9a01a-bundle\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:21:24.303173 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:24.303118 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77a02f88-756d-4080-b5bc-eba068e9a01a-util\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:21:24.303173 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:24.303131 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbkg2\" (UniqueName: \"kubernetes.io/projected/77a02f88-756d-4080-b5bc-eba068e9a01a-kube-api-access-gbkg2\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:21:24.969851 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:24.969824 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95" Jul 02 19:21:24.970037 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:24.969820 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffgf95" event={"ID":"77a02f88-756d-4080-b5bc-eba068e9a01a","Type":"ContainerDied","Data":"aa8c892743f963db58d681ab1be0a505c53e5133f4de306e5a2afdbe38edc7ae"} Jul 02 19:21:24.970037 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:24.969952 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa8c892743f963db58d681ab1be0a505c53e5133f4de306e5a2afdbe38edc7ae" Jul 02 19:21:25.976664 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:25.976628 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5zpnd" event={"ID":"644ac5a6-9bc9-4226-a55a-1a1b1e4505a8","Type":"ContainerStarted","Data":"0d2ded4d557a29ea9adaf133960f5dd3361f011010c8ccf5e3d29487f135eba7"} Jul 02 19:21:25.991921 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:25.991869 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5zpnd" podStartSLOduration=1.593322189 podStartE2EDuration="5.991854211s" totalImagesPullingTime="4.398532022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:21:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:21:25.990571452 +0000 UTC m=+598.537296477" watchObservedRunningTime="2026-07-02 19:21:25.991854211 +0000 UTC m=+598.538579217" Jul 02 19:21:30.985060 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:30.985013 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rm5fh"] Jul 02 19:21:30.985435 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:30.985328 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77a02f88-756d-4080-b5bc-eba068e9a01a" containerName="util" Jul 02 19:21:30.985435 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:30.985342 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="77a02f88-756d-4080-b5bc-eba068e9a01a" containerName="util" Jul 02 19:21:30.985435 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:30.985350 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77a02f88-756d-4080-b5bc-eba068e9a01a" containerName="extract" Jul 02 19:21:30.985435 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:30.985357 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="77a02f88-756d-4080-b5bc-eba068e9a01a" containerName="extract" Jul 02 19:21:30.985435 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:30.985369 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77a02f88-756d-4080-b5bc-eba068e9a01a" containerName="pull" Jul 02 19:21:30.985435 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:30.985376 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="77a02f88-756d-4080-b5bc-eba068e9a01a" containerName="pull" Jul 02 19:21:30.985435 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:30.985422 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="77a02f88-756d-4080-b5bc-eba068e9a01a" containerName="extract" Jul 02 19:21:30.988227 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:30.988209 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rm5fh" Jul 02 19:21:30.991209 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:30.991187 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-csf2l\"" Jul 02 19:21:30.999733 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:30.999713 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rm5fh"] Jul 02 19:21:31.056915 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:31.056879 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/993b6083-7cd7-4601-9a82-c171e6036ce1-bound-sa-token\") pod \"cert-manager-55b4d954d5-rm5fh\" (UID: \"993b6083-7cd7-4601-9a82-c171e6036ce1\") " pod="cert-manager/cert-manager-55b4d954d5-rm5fh" Jul 02 19:21:31.057088 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:31.056933 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxv4j\" (UniqueName: \"kubernetes.io/projected/993b6083-7cd7-4601-9a82-c171e6036ce1-kube-api-access-mxv4j\") pod \"cert-manager-55b4d954d5-rm5fh\" (UID: \"993b6083-7cd7-4601-9a82-c171e6036ce1\") " pod="cert-manager/cert-manager-55b4d954d5-rm5fh" Jul 02 19:21:31.157867 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:31.157833 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/993b6083-7cd7-4601-9a82-c171e6036ce1-bound-sa-token\") pod \"cert-manager-55b4d954d5-rm5fh\" (UID: \"993b6083-7cd7-4601-9a82-c171e6036ce1\") " pod="cert-manager/cert-manager-55b4d954d5-rm5fh" Jul 02 19:21:31.158015 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:31.157892 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxv4j\" (UniqueName: \"kubernetes.io/projected/993b6083-7cd7-4601-9a82-c171e6036ce1-kube-api-access-mxv4j\") pod \"cert-manager-55b4d954d5-rm5fh\" (UID: \"993b6083-7cd7-4601-9a82-c171e6036ce1\") " pod="cert-manager/cert-manager-55b4d954d5-rm5fh" Jul 02 19:21:31.166423 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:31.166394 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxv4j\" (UniqueName: \"kubernetes.io/projected/993b6083-7cd7-4601-9a82-c171e6036ce1-kube-api-access-mxv4j\") pod \"cert-manager-55b4d954d5-rm5fh\" (UID: \"993b6083-7cd7-4601-9a82-c171e6036ce1\") " pod="cert-manager/cert-manager-55b4d954d5-rm5fh" Jul 02 19:21:31.166797 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:31.166780 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/993b6083-7cd7-4601-9a82-c171e6036ce1-bound-sa-token\") pod \"cert-manager-55b4d954d5-rm5fh\" (UID: \"993b6083-7cd7-4601-9a82-c171e6036ce1\") " pod="cert-manager/cert-manager-55b4d954d5-rm5fh" Jul 02 19:21:31.297769 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:31.297695 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rm5fh" Jul 02 19:21:31.423964 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:31.423919 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rm5fh"] Jul 02 19:21:31.425895 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:21:31.425865 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod993b6083_7cd7_4601_9a82_c171e6036ce1.slice/crio-13cca6ab02b16d67fa773ec645d9bb0ef542fae0e0eb3fb4e83fd84ad3304984 WatchSource:0}: Error finding container 13cca6ab02b16d67fa773ec645d9bb0ef542fae0e0eb3fb4e83fd84ad3304984: Status 404 returned error can't find the container with id 13cca6ab02b16d67fa773ec645d9bb0ef542fae0e0eb3fb4e83fd84ad3304984 Jul 02 19:21:31.999568 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:31.999529 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rm5fh" event={"ID":"993b6083-7cd7-4601-9a82-c171e6036ce1","Type":"ContainerStarted","Data":"27ad2518a67c0dd420cb7b26a66df7b5e8e96d1a84ff91ddadc2a9e416450fd2"} Jul 02 19:21:31.999568 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:31.999575 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rm5fh" event={"ID":"993b6083-7cd7-4601-9a82-c171e6036ce1","Type":"ContainerStarted","Data":"13cca6ab02b16d67fa773ec645d9bb0ef542fae0e0eb3fb4e83fd84ad3304984"} Jul 02 19:21:32.017115 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:32.017070 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-rm5fh" podStartSLOduration=2.017058529 podStartE2EDuration="2.017058529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:21:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:21:32.016558651 +0000 UTC m=+604.563283669" watchObservedRunningTime="2026-07-02 19:21:32.017058529 +0000 UTC m=+604.563783536" Jul 02 19:21:39.085611 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:39.085567 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z"] Jul 02 19:21:39.136030 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:39.135993 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z"] Jul 02 19:21:39.136188 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:39.136123 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z" Jul 02 19:21:39.139344 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:39.139323 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 19:21:39.140142 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:39.140125 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 19:21:39.140246 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:39.140129 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bkmn9\"" Jul 02 19:21:39.221357 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:39.221318 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e08888fe-1e40-430b-b713-d02430905c43-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z\" (UID: \"e08888fe-1e40-430b-b713-d02430905c43\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z" Jul 02 19:21:39.221357 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:39.221361 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e08888fe-1e40-430b-b713-d02430905c43-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z\" (UID: \"e08888fe-1e40-430b-b713-d02430905c43\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z" Jul 02 19:21:39.221618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:39.221389 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-599wv\" (UniqueName: \"kubernetes.io/projected/e08888fe-1e40-430b-b713-d02430905c43-kube-api-access-599wv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z\" (UID: \"e08888fe-1e40-430b-b713-d02430905c43\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z" Jul 02 19:21:39.322100 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:39.322069 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e08888fe-1e40-430b-b713-d02430905c43-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z\" (UID: \"e08888fe-1e40-430b-b713-d02430905c43\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z" Jul 02 19:21:39.322100 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:39.322108 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e08888fe-1e40-430b-b713-d02430905c43-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z\" (UID: \"e08888fe-1e40-430b-b713-d02430905c43\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z" Jul 02 19:21:39.322362 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:39.322126 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-599wv\" (UniqueName: \"kubernetes.io/projected/e08888fe-1e40-430b-b713-d02430905c43-kube-api-access-599wv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z\" (UID: \"e08888fe-1e40-430b-b713-d02430905c43\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z" Jul 02 19:21:39.322529 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:39.322506 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e08888fe-1e40-430b-b713-d02430905c43-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z\" (UID: \"e08888fe-1e40-430b-b713-d02430905c43\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z" Jul 02 19:21:39.322601 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:39.322549 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e08888fe-1e40-430b-b713-d02430905c43-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z\" (UID: \"e08888fe-1e40-430b-b713-d02430905c43\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z" Jul 02 19:21:39.330304 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:39.330272 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-599wv\" (UniqueName: \"kubernetes.io/projected/e08888fe-1e40-430b-b713-d02430905c43-kube-api-access-599wv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z\" (UID: \"e08888fe-1e40-430b-b713-d02430905c43\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z" Jul 02 19:21:39.445394 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:39.445359 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z" Jul 02 19:21:39.591433 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:39.591400 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z"] Jul 02 19:21:39.593820 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:21:39.593792 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode08888fe_1e40_430b_b713_d02430905c43.slice/crio-d4c6adaa877aae476348c900e1fa98cac708b57a4c7fdc7115fe97acc74d62ab WatchSource:0}: Error finding container d4c6adaa877aae476348c900e1fa98cac708b57a4c7fdc7115fe97acc74d62ab: Status 404 returned error can't find the container with id d4c6adaa877aae476348c900e1fa98cac708b57a4c7fdc7115fe97acc74d62ab Jul 02 19:21:40.030843 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:40.030761 2556 generic.go:356] "Generic (PLEG): container finished" podID="e08888fe-1e40-430b-b713-d02430905c43" containerID="cc7851f8f70dba5205b234b6547c2d133cf3026412d23f53dce8baf560b80e83" exitCode=0 Jul 02 19:21:40.030843 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:40.030812 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z" event={"ID":"e08888fe-1e40-430b-b713-d02430905c43","Type":"ContainerDied","Data":"cc7851f8f70dba5205b234b6547c2d133cf3026412d23f53dce8baf560b80e83"} Jul 02 19:21:40.030843 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:40.030845 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z" event={"ID":"e08888fe-1e40-430b-b713-d02430905c43","Type":"ContainerStarted","Data":"d4c6adaa877aae476348c900e1fa98cac708b57a4c7fdc7115fe97acc74d62ab"} Jul 02 19:21:42.040062 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:42.040023 2556 generic.go:356] "Generic (PLEG): container finished" podID="e08888fe-1e40-430b-b713-d02430905c43" containerID="192324e12d2ed9e26058a30f7de75fa46851fbec2b476a34ee97da2b1e8693f7" exitCode=0 Jul 02 19:21:42.040420 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:42.040070 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z" event={"ID":"e08888fe-1e40-430b-b713-d02430905c43","Type":"ContainerDied","Data":"192324e12d2ed9e26058a30f7de75fa46851fbec2b476a34ee97da2b1e8693f7"} Jul 02 19:21:43.045868 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:43.045831 2556 generic.go:356] "Generic (PLEG): container finished" podID="e08888fe-1e40-430b-b713-d02430905c43" containerID="bb17fec2d6e84ff62a90f19e3ed4f3fcd1c04ba21cd68f6b4b583d03e7509c77" exitCode=0 Jul 02 19:21:43.046257 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:43.045885 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z" event={"ID":"e08888fe-1e40-430b-b713-d02430905c43","Type":"ContainerDied","Data":"bb17fec2d6e84ff62a90f19e3ed4f3fcd1c04ba21cd68f6b4b583d03e7509c77"} Jul 02 19:21:44.184638 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:44.184608 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z" Jul 02 19:21:44.364471 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:44.364390 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e08888fe-1e40-430b-b713-d02430905c43-bundle\") pod \"e08888fe-1e40-430b-b713-d02430905c43\" (UID: \"e08888fe-1e40-430b-b713-d02430905c43\") " Jul 02 19:21:44.364471 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:44.364425 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-599wv\" (UniqueName: \"kubernetes.io/projected/e08888fe-1e40-430b-b713-d02430905c43-kube-api-access-599wv\") pod \"e08888fe-1e40-430b-b713-d02430905c43\" (UID: \"e08888fe-1e40-430b-b713-d02430905c43\") " Jul 02 19:21:44.364471 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:44.364454 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e08888fe-1e40-430b-b713-d02430905c43-util\") pod \"e08888fe-1e40-430b-b713-d02430905c43\" (UID: \"e08888fe-1e40-430b-b713-d02430905c43\") " Jul 02 19:21:44.365235 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:44.365212 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e08888fe-1e40-430b-b713-d02430905c43-bundle" (OuterVolumeSpecName: "bundle") pod "e08888fe-1e40-430b-b713-d02430905c43" (UID: "e08888fe-1e40-430b-b713-d02430905c43"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:21:44.366625 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:44.366598 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e08888fe-1e40-430b-b713-d02430905c43-kube-api-access-599wv" (OuterVolumeSpecName: "kube-api-access-599wv") pod "e08888fe-1e40-430b-b713-d02430905c43" (UID: "e08888fe-1e40-430b-b713-d02430905c43"). InnerVolumeSpecName "kube-api-access-599wv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:21:44.369873 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:44.369846 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e08888fe-1e40-430b-b713-d02430905c43-util" (OuterVolumeSpecName: "util") pod "e08888fe-1e40-430b-b713-d02430905c43" (UID: "e08888fe-1e40-430b-b713-d02430905c43"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:21:44.465274 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:44.465242 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e08888fe-1e40-430b-b713-d02430905c43-bundle\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:21:44.465274 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:44.465269 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-599wv\" (UniqueName: \"kubernetes.io/projected/e08888fe-1e40-430b-b713-d02430905c43-kube-api-access-599wv\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:21:44.465274 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:44.465280 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e08888fe-1e40-430b-b713-d02430905c43-util\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:21:45.055466 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:45.055427 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z" event={"ID":"e08888fe-1e40-430b-b713-d02430905c43","Type":"ContainerDied","Data":"d4c6adaa877aae476348c900e1fa98cac708b57a4c7fdc7115fe97acc74d62ab"} Jul 02 19:21:45.055632 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:45.055472 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnwj9z" Jul 02 19:21:45.055632 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:45.055476 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4c6adaa877aae476348c900e1fa98cac708b57a4c7fdc7115fe97acc74d62ab" Jul 02 19:21:50.539665 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.539586 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk"] Jul 02 19:21:50.540124 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.539818 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e08888fe-1e40-430b-b713-d02430905c43" containerName="util" Jul 02 19:21:50.540124 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.539828 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e08888fe-1e40-430b-b713-d02430905c43" containerName="util" Jul 02 19:21:50.540124 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.539837 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e08888fe-1e40-430b-b713-d02430905c43" containerName="pull" Jul 02 19:21:50.540124 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.539842 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e08888fe-1e40-430b-b713-d02430905c43" containerName="pull" Jul 02 19:21:50.540124 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.539850 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e08888fe-1e40-430b-b713-d02430905c43" containerName="extract" Jul 02 19:21:50.540124 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.539856 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e08888fe-1e40-430b-b713-d02430905c43" containerName="extract" Jul 02 19:21:50.540124 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.539904 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e08888fe-1e40-430b-b713-d02430905c43" containerName="extract" Jul 02 19:21:50.542341 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.542325 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk" Jul 02 19:21:50.545723 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.545699 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 02 19:21:50.545893 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.545850 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 02 19:21:50.545963 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.545912 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 02 19:21:50.547023 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.547002 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-krjss\"" Jul 02 19:21:50.547182 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.547040 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 02 19:21:50.547274 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.547260 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 02 19:21:50.560648 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.560624 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk"] Jul 02 19:21:50.709071 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.709030 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de1ba6eb-2ad0-431d-a1e1-3211cb86cee4-cert\") pod \"lws-controller-manager-5d5f586fb-gt8gk\" (UID: \"de1ba6eb-2ad0-431d-a1e1-3211cb86cee4\") " pod="openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk" Jul 02 19:21:50.709071 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.709071 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/de1ba6eb-2ad0-431d-a1e1-3211cb86cee4-manager-config\") pod \"lws-controller-manager-5d5f586fb-gt8gk\" (UID: \"de1ba6eb-2ad0-431d-a1e1-3211cb86cee4\") " pod="openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk" Jul 02 19:21:50.709355 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.709137 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppbm8\" (UniqueName: \"kubernetes.io/projected/de1ba6eb-2ad0-431d-a1e1-3211cb86cee4-kube-api-access-ppbm8\") pod \"lws-controller-manager-5d5f586fb-gt8gk\" (UID: \"de1ba6eb-2ad0-431d-a1e1-3211cb86cee4\") " pod="openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk" Jul 02 19:21:50.709355 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.709184 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/de1ba6eb-2ad0-431d-a1e1-3211cb86cee4-metrics-cert\") pod \"lws-controller-manager-5d5f586fb-gt8gk\" (UID: \"de1ba6eb-2ad0-431d-a1e1-3211cb86cee4\") " pod="openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk" Jul 02 19:21:50.810302 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.810197 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppbm8\" (UniqueName: \"kubernetes.io/projected/de1ba6eb-2ad0-431d-a1e1-3211cb86cee4-kube-api-access-ppbm8\") pod \"lws-controller-manager-5d5f586fb-gt8gk\" (UID: \"de1ba6eb-2ad0-431d-a1e1-3211cb86cee4\") " pod="openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk" Jul 02 19:21:50.810302 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.810251 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/de1ba6eb-2ad0-431d-a1e1-3211cb86cee4-metrics-cert\") pod \"lws-controller-manager-5d5f586fb-gt8gk\" (UID: \"de1ba6eb-2ad0-431d-a1e1-3211cb86cee4\") " pod="openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk" Jul 02 19:21:50.810302 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.810299 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de1ba6eb-2ad0-431d-a1e1-3211cb86cee4-cert\") pod \"lws-controller-manager-5d5f586fb-gt8gk\" (UID: \"de1ba6eb-2ad0-431d-a1e1-3211cb86cee4\") " pod="openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk" Jul 02 19:21:50.810544 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.810324 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/de1ba6eb-2ad0-431d-a1e1-3211cb86cee4-manager-config\") pod \"lws-controller-manager-5d5f586fb-gt8gk\" (UID: \"de1ba6eb-2ad0-431d-a1e1-3211cb86cee4\") " pod="openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk" Jul 02 19:21:50.811056 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.811032 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/de1ba6eb-2ad0-431d-a1e1-3211cb86cee4-manager-config\") pod \"lws-controller-manager-5d5f586fb-gt8gk\" (UID: \"de1ba6eb-2ad0-431d-a1e1-3211cb86cee4\") " pod="openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk" Jul 02 19:21:50.813784 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.813229 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de1ba6eb-2ad0-431d-a1e1-3211cb86cee4-cert\") pod \"lws-controller-manager-5d5f586fb-gt8gk\" (UID: \"de1ba6eb-2ad0-431d-a1e1-3211cb86cee4\") " pod="openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk" Jul 02 19:21:50.813784 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.813242 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/de1ba6eb-2ad0-431d-a1e1-3211cb86cee4-metrics-cert\") pod \"lws-controller-manager-5d5f586fb-gt8gk\" (UID: \"de1ba6eb-2ad0-431d-a1e1-3211cb86cee4\") " pod="openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk" Jul 02 19:21:50.839733 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.839702 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppbm8\" (UniqueName: \"kubernetes.io/projected/de1ba6eb-2ad0-431d-a1e1-3211cb86cee4-kube-api-access-ppbm8\") pod \"lws-controller-manager-5d5f586fb-gt8gk\" (UID: \"de1ba6eb-2ad0-431d-a1e1-3211cb86cee4\") " pod="openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk" Jul 02 19:21:50.851564 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:50.851221 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk" Jul 02 19:21:51.004599 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:51.004575 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk"] Jul 02 19:21:51.006946 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:21:51.006919 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde1ba6eb_2ad0_431d_a1e1_3211cb86cee4.slice/crio-a3329a974b4a0966298c68dac102c89ed540faf9d7ce5c471d27ebe18570db10 WatchSource:0}: Error finding container a3329a974b4a0966298c68dac102c89ed540faf9d7ce5c471d27ebe18570db10: Status 404 returned error can't find the container with id a3329a974b4a0966298c68dac102c89ed540faf9d7ce5c471d27ebe18570db10 Jul 02 19:21:51.009073 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:51.009052 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 19:21:51.078026 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:51.077939 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk" event={"ID":"de1ba6eb-2ad0-431d-a1e1-3211cb86cee4","Type":"ContainerStarted","Data":"a3329a974b4a0966298c68dac102c89ed540faf9d7ce5c471d27ebe18570db10"} Jul 02 19:21:51.918929 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:51.918898 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9"] Jul 02 19:21:51.921733 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:51.921713 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" Jul 02 19:21:51.924494 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:51.924471 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 02 19:21:51.924640 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:51.924522 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-s29m5\"" Jul 02 19:21:51.924784 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:51.924766 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 02 19:21:51.924920 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:51.924900 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 02 19:21:51.924988 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:51.924971 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 02 19:21:51.936074 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:51.936047 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9"] Jul 02 19:21:52.019978 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:52.019942 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/396a53d7-c019-4512-a7ea-e7ab9ed0f53d-webhook-cert\") pod \"opendatahub-operator-controller-manager-6974d4f959-5hkj9\" (UID: \"396a53d7-c019-4512-a7ea-e7ab9ed0f53d\") " pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" Jul 02 19:21:52.020175 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:52.019991 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/396a53d7-c019-4512-a7ea-e7ab9ed0f53d-manifests\") pod \"opendatahub-operator-controller-manager-6974d4f959-5hkj9\" (UID: \"396a53d7-c019-4512-a7ea-e7ab9ed0f53d\") " pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" Jul 02 19:21:52.020175 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:52.020021 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/396a53d7-c019-4512-a7ea-e7ab9ed0f53d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6974d4f959-5hkj9\" (UID: \"396a53d7-c019-4512-a7ea-e7ab9ed0f53d\") " pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" Jul 02 19:21:52.020175 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:52.020104 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/396a53d7-c019-4512-a7ea-e7ab9ed0f53d-tmp\") pod \"opendatahub-operator-controller-manager-6974d4f959-5hkj9\" (UID: \"396a53d7-c019-4512-a7ea-e7ab9ed0f53d\") " pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" Jul 02 19:21:52.020175 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:52.020135 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98kgf\" (UniqueName: \"kubernetes.io/projected/396a53d7-c019-4512-a7ea-e7ab9ed0f53d-kube-api-access-98kgf\") pod \"opendatahub-operator-controller-manager-6974d4f959-5hkj9\" (UID: \"396a53d7-c019-4512-a7ea-e7ab9ed0f53d\") " pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" Jul 02 19:21:52.121324 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:52.121293 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/396a53d7-c019-4512-a7ea-e7ab9ed0f53d-webhook-cert\") pod \"opendatahub-operator-controller-manager-6974d4f959-5hkj9\" (UID: \"396a53d7-c019-4512-a7ea-e7ab9ed0f53d\") " pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" Jul 02 19:21:52.121503 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:52.121336 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/396a53d7-c019-4512-a7ea-e7ab9ed0f53d-manifests\") pod \"opendatahub-operator-controller-manager-6974d4f959-5hkj9\" (UID: \"396a53d7-c019-4512-a7ea-e7ab9ed0f53d\") " pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" Jul 02 19:21:52.121503 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:52.121357 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/396a53d7-c019-4512-a7ea-e7ab9ed0f53d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6974d4f959-5hkj9\" (UID: \"396a53d7-c019-4512-a7ea-e7ab9ed0f53d\") " pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" Jul 02 19:21:52.121503 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:52.121434 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/396a53d7-c019-4512-a7ea-e7ab9ed0f53d-tmp\") pod \"opendatahub-operator-controller-manager-6974d4f959-5hkj9\" (UID: \"396a53d7-c019-4512-a7ea-e7ab9ed0f53d\") " pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" Jul 02 19:21:52.121503 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:52.121455 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98kgf\" (UniqueName: \"kubernetes.io/projected/396a53d7-c019-4512-a7ea-e7ab9ed0f53d-kube-api-access-98kgf\") pod \"opendatahub-operator-controller-manager-6974d4f959-5hkj9\" (UID: \"396a53d7-c019-4512-a7ea-e7ab9ed0f53d\") " pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" Jul 02 19:21:52.121757 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:52.121726 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/396a53d7-c019-4512-a7ea-e7ab9ed0f53d-manifests\") pod \"opendatahub-operator-controller-manager-6974d4f959-5hkj9\" (UID: \"396a53d7-c019-4512-a7ea-e7ab9ed0f53d\") " pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" Jul 02 19:21:52.123742 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:52.123710 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/396a53d7-c019-4512-a7ea-e7ab9ed0f53d-tmp\") pod \"opendatahub-operator-controller-manager-6974d4f959-5hkj9\" (UID: \"396a53d7-c019-4512-a7ea-e7ab9ed0f53d\") " pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" Jul 02 19:21:52.123908 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:52.123892 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/396a53d7-c019-4512-a7ea-e7ab9ed0f53d-webhook-cert\") pod \"opendatahub-operator-controller-manager-6974d4f959-5hkj9\" (UID: \"396a53d7-c019-4512-a7ea-e7ab9ed0f53d\") " pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" Jul 02 19:21:52.123994 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:52.123972 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/396a53d7-c019-4512-a7ea-e7ab9ed0f53d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6974d4f959-5hkj9\" (UID: \"396a53d7-c019-4512-a7ea-e7ab9ed0f53d\") " pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" Jul 02 19:21:52.130600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:52.130424 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98kgf\" (UniqueName: \"kubernetes.io/projected/396a53d7-c019-4512-a7ea-e7ab9ed0f53d-kube-api-access-98kgf\") pod \"opendatahub-operator-controller-manager-6974d4f959-5hkj9\" (UID: \"396a53d7-c019-4512-a7ea-e7ab9ed0f53d\") " pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" Jul 02 19:21:52.232555 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:52.232469 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" Jul 02 19:21:52.385935 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:52.385896 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9"] Jul 02 19:21:52.389263 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:21:52.389233 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod396a53d7_c019_4512_a7ea_e7ab9ed0f53d.slice/crio-a8cb6a14e2fdc3d45131aa27fab62ee52032ff38043dfba6d93e30a7beba916d WatchSource:0}: Error finding container a8cb6a14e2fdc3d45131aa27fab62ee52032ff38043dfba6d93e30a7beba916d: Status 404 returned error can't find the container with id a8cb6a14e2fdc3d45131aa27fab62ee52032ff38043dfba6d93e30a7beba916d Jul 02 19:21:53.090470 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:53.090433 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" event={"ID":"396a53d7-c019-4512-a7ea-e7ab9ed0f53d","Type":"ContainerStarted","Data":"a8cb6a14e2fdc3d45131aa27fab62ee52032ff38043dfba6d93e30a7beba916d"} Jul 02 19:21:53.091882 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:53.091854 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk" event={"ID":"de1ba6eb-2ad0-431d-a1e1-3211cb86cee4","Type":"ContainerStarted","Data":"da18802ef1745530311d4e20151c41f7993a246070bc17f29e18ce71bb42982b"} Jul 02 19:21:53.092032 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:53.091959 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk" Jul 02 19:21:53.110472 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:53.110421 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk" podStartSLOduration=1.154706241 podStartE2EDuration="3.110406544s" totalImagesPullingTime="1.955700303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:21:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:21:53.109399617 +0000 UTC m=+625.656124638" watchObservedRunningTime="2026-07-02 19:21:53.110406544 +0000 UTC m=+625.657131550" Jul 02 19:21:56.107829 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:56.107795 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" event={"ID":"396a53d7-c019-4512-a7ea-e7ab9ed0f53d","Type":"ContainerStarted","Data":"760046e5c8558fdc620feb244ef8eb75b8ccdbd535cc3ea0743923c704f8599b"} Jul 02 19:21:57.113478 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:57.113390 2556 generic.go:356] "Generic (PLEG): container finished" podID="396a53d7-c019-4512-a7ea-e7ab9ed0f53d" containerID="760046e5c8558fdc620feb244ef8eb75b8ccdbd535cc3ea0743923c704f8599b" exitCode=0 Jul 02 19:21:57.113478 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:57.113459 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" event={"ID":"396a53d7-c019-4512-a7ea-e7ab9ed0f53d","Type":"ContainerDied","Data":"760046e5c8558fdc620feb244ef8eb75b8ccdbd535cc3ea0743923c704f8599b"} Jul 02 19:21:58.120250 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:58.120207 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" event={"ID":"396a53d7-c019-4512-a7ea-e7ab9ed0f53d","Type":"ContainerStarted","Data":"295a42a0f8358b42ef89892b0d236064450fa0b71bf0f7eefecde7c985d83955"} Jul 02 19:21:58.120676 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:58.120334 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" Jul 02 19:21:58.143362 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:21:58.143312 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" podStartSLOduration=2.499572102 podStartE2EDuration="7.143296274s" totalImagesPullingTime="3.600120267s" totalInitContainerRuntime="1.043603905s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:21:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:21:58.14286952 +0000 UTC m=+630.689594549" watchObservedRunningTime="2026-07-02 19:21:58.143296274 +0000 UTC m=+630.690021281" Jul 02 19:22:04.099620 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:04.099583 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5d5f586fb-gt8gk" Jul 02 19:22:09.126764 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:09.126733 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6974d4f959-5hkj9" Jul 02 19:22:24.735118 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.732550 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8"] Jul 02 19:22:24.738275 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.738243 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:24.741086 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.741062 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 02 19:22:24.741268 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.741095 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 02 19:22:24.741268 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.741107 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-q2lll\"" Jul 02 19:22:24.741268 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.741094 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 02 19:22:24.741405 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.741308 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 02 19:22:24.741543 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.741527 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 02 19:22:24.742198 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.742183 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 02 19:22:24.749695 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.749649 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8"] Jul 02 19:22:24.876009 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.875973 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/58bcf208-b1c1-4b43-ba8d-1877473c0582-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:24.876212 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.876018 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/58bcf208-b1c1-4b43-ba8d-1877473c0582-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:24.876212 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.876050 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/58bcf208-b1c1-4b43-ba8d-1877473c0582-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:24.876212 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.876115 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/58bcf208-b1c1-4b43-ba8d-1877473c0582-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:24.876212 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.876170 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58bcf208-b1c1-4b43-ba8d-1877473c0582-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:24.876212 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.876196 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtq7w\" (UniqueName: \"kubernetes.io/projected/58bcf208-b1c1-4b43-ba8d-1877473c0582-kube-api-access-vtq7w\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:24.876377 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.876216 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/58bcf208-b1c1-4b43-ba8d-1877473c0582-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:24.977092 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.977058 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/58bcf208-b1c1-4b43-ba8d-1877473c0582-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:24.977092 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.977096 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58bcf208-b1c1-4b43-ba8d-1877473c0582-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:24.977333 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.977128 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtq7w\" (UniqueName: \"kubernetes.io/projected/58bcf208-b1c1-4b43-ba8d-1877473c0582-kube-api-access-vtq7w\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:24.977333 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.977171 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/58bcf208-b1c1-4b43-ba8d-1877473c0582-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:24.977333 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.977221 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/58bcf208-b1c1-4b43-ba8d-1877473c0582-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:24.977333 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.977253 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/58bcf208-b1c1-4b43-ba8d-1877473c0582-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:24.977333 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.977316 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/58bcf208-b1c1-4b43-ba8d-1877473c0582-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:24.977940 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.977910 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/58bcf208-b1c1-4b43-ba8d-1877473c0582-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:24.979562 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.979534 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/58bcf208-b1c1-4b43-ba8d-1877473c0582-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:24.979562 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.979555 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/58bcf208-b1c1-4b43-ba8d-1877473c0582-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:24.979678 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.979582 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/58bcf208-b1c1-4b43-ba8d-1877473c0582-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:24.979678 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:24.979667 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/58bcf208-b1c1-4b43-ba8d-1877473c0582-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:25.001444 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:25.001386 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58bcf208-b1c1-4b43-ba8d-1877473c0582-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:25.008843 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:25.008817 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtq7w\" (UniqueName: \"kubernetes.io/projected/58bcf208-b1c1-4b43-ba8d-1877473c0582-kube-api-access-vtq7w\") pod \"istiod-openshift-gateway-94bb8fbfd-vhsn8\" (UID: \"58bcf208-b1c1-4b43-ba8d-1877473c0582\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:25.048298 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:25.048269 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:25.195083 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:25.195065 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8"] Jul 02 19:22:25.197679 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:22:25.197650 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58bcf208_b1c1_4b43_ba8d_1877473c0582.slice/crio-dc4c11c0e5cc2106ff1f108f270064a03565bd9775b9a900f777c6b8c9a9ae3a WatchSource:0}: Error finding container dc4c11c0e5cc2106ff1f108f270064a03565bd9775b9a900f777c6b8c9a9ae3a: Status 404 returned error can't find the container with id dc4c11c0e5cc2106ff1f108f270064a03565bd9775b9a900f777c6b8c9a9ae3a Jul 02 19:22:25.240998 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:25.240958 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" event={"ID":"58bcf208-b1c1-4b43-ba8d-1877473c0582","Type":"ContainerStarted","Data":"dc4c11c0e5cc2106ff1f108f270064a03565bd9775b9a900f777c6b8c9a9ae3a"} Jul 02 19:22:28.400176 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:28.400123 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:22:28.400528 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:28.400219 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:22:29.260582 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:29.260546 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" event={"ID":"58bcf208-b1c1-4b43-ba8d-1877473c0582","Type":"ContainerStarted","Data":"ed4b1a47cb3f9d30c4aeb03850d273f47803fbe1e1b81cfeb15c860ead9a9fa1"} Jul 02 19:22:29.260774 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:29.260751 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:29.262519 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:29.262495 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" Jul 02 19:22:29.299214 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:22:29.299161 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vhsn8" podStartSLOduration=2.099138619 podStartE2EDuration="5.299130076s" totalImagesPullingTime="3.199991457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:22:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:22:29.296859913 +0000 UTC m=+661.843584958" watchObservedRunningTime="2026-07-02 19:22:29.299130076 +0000 UTC m=+661.845855083" Jul 02 19:23:29.240828 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.240787 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65"] Jul 02 19:23:29.243891 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.243874 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65" Jul 02 19:23:29.246335 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.246311 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zd8cf\"" Jul 02 19:23:29.246466 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.246349 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 19:23:29.247467 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.247453 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 19:23:29.252037 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.252017 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65"] Jul 02 19:23:29.292947 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.292907 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edfcfd82-124a-4ff2-be30-a65941f85ce5-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65\" (UID: \"edfcfd82-124a-4ff2-be30-a65941f85ce5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65" Jul 02 19:23:29.293107 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.292958 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bszfm\" (UniqueName: \"kubernetes.io/projected/edfcfd82-124a-4ff2-be30-a65941f85ce5-kube-api-access-bszfm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65\" (UID: \"edfcfd82-124a-4ff2-be30-a65941f85ce5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65" Jul 02 19:23:29.293107 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.292989 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edfcfd82-124a-4ff2-be30-a65941f85ce5-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65\" (UID: \"edfcfd82-124a-4ff2-be30-a65941f85ce5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65" Jul 02 19:23:29.394125 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.394081 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edfcfd82-124a-4ff2-be30-a65941f85ce5-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65\" (UID: \"edfcfd82-124a-4ff2-be30-a65941f85ce5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65" Jul 02 19:23:29.394322 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.394137 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bszfm\" (UniqueName: \"kubernetes.io/projected/edfcfd82-124a-4ff2-be30-a65941f85ce5-kube-api-access-bszfm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65\" (UID: \"edfcfd82-124a-4ff2-be30-a65941f85ce5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65" Jul 02 19:23:29.394322 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.394190 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edfcfd82-124a-4ff2-be30-a65941f85ce5-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65\" (UID: \"edfcfd82-124a-4ff2-be30-a65941f85ce5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65" Jul 02 19:23:29.394571 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.394552 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edfcfd82-124a-4ff2-be30-a65941f85ce5-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65\" (UID: \"edfcfd82-124a-4ff2-be30-a65941f85ce5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65" Jul 02 19:23:29.394605 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.394557 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edfcfd82-124a-4ff2-be30-a65941f85ce5-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65\" (UID: \"edfcfd82-124a-4ff2-be30-a65941f85ce5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65" Jul 02 19:23:29.403669 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.403643 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bszfm\" (UniqueName: \"kubernetes.io/projected/edfcfd82-124a-4ff2-be30-a65941f85ce5-kube-api-access-bszfm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65\" (UID: \"edfcfd82-124a-4ff2-be30-a65941f85ce5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65" Jul 02 19:23:29.553844 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.553758 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65" Jul 02 19:23:29.681441 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.681414 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65"] Jul 02 19:23:29.683482 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:23:29.683459 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedfcfd82_124a_4ff2_be30_a65941f85ce5.slice/crio-b06857f3c786fc40342cd3e98331f50b101a585857817b5e4bae6f43e7a9a4c5 WatchSource:0}: Error finding container b06857f3c786fc40342cd3e98331f50b101a585857817b5e4bae6f43e7a9a4c5: Status 404 returned error can't find the container with id b06857f3c786fc40342cd3e98331f50b101a585857817b5e4bae6f43e7a9a4c5 Jul 02 19:23:29.842439 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.842358 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq"] Jul 02 19:23:29.845564 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.845548 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq" Jul 02 19:23:29.854044 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.854021 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq"] Jul 02 19:23:29.897301 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.897258 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c48642e-167b-466b-91a4-08490e74cf80-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq\" (UID: \"1c48642e-167b-466b-91a4-08490e74cf80\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq" Jul 02 19:23:29.897464 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.897314 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hldpr\" (UniqueName: \"kubernetes.io/projected/1c48642e-167b-466b-91a4-08490e74cf80-kube-api-access-hldpr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq\" (UID: \"1c48642e-167b-466b-91a4-08490e74cf80\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq" Jul 02 19:23:29.897464 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.897367 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c48642e-167b-466b-91a4-08490e74cf80-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq\" (UID: \"1c48642e-167b-466b-91a4-08490e74cf80\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq" Jul 02 19:23:29.998765 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.998729 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c48642e-167b-466b-91a4-08490e74cf80-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq\" (UID: \"1c48642e-167b-466b-91a4-08490e74cf80\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq" Jul 02 19:23:29.998765 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.998770 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hldpr\" (UniqueName: \"kubernetes.io/projected/1c48642e-167b-466b-91a4-08490e74cf80-kube-api-access-hldpr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq\" (UID: \"1c48642e-167b-466b-91a4-08490e74cf80\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq" Jul 02 19:23:29.998989 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.998802 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c48642e-167b-466b-91a4-08490e74cf80-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq\" (UID: \"1c48642e-167b-466b-91a4-08490e74cf80\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq" Jul 02 19:23:29.999137 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.999117 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c48642e-167b-466b-91a4-08490e74cf80-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq\" (UID: \"1c48642e-167b-466b-91a4-08490e74cf80\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq" Jul 02 19:23:29.999219 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:29.999191 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c48642e-167b-466b-91a4-08490e74cf80-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq\" (UID: \"1c48642e-167b-466b-91a4-08490e74cf80\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq" Jul 02 19:23:30.007578 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.007554 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hldpr\" (UniqueName: \"kubernetes.io/projected/1c48642e-167b-466b-91a4-08490e74cf80-kube-api-access-hldpr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq\" (UID: \"1c48642e-167b-466b-91a4-08490e74cf80\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq" Jul 02 19:23:30.154881 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.154793 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq" Jul 02 19:23:30.241600 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.241569 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5"] Jul 02 19:23:30.245140 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.245116 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5" Jul 02 19:23:30.253319 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.253105 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5"] Jul 02 19:23:30.284115 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.284082 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq"] Jul 02 19:23:30.286932 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:23:30.286910 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c48642e_167b_466b_91a4_08490e74cf80.slice/crio-334b02302cd976aeb1d6f6acc9f20d8fcfa772e3789b77756dcf5acb9c1540cb WatchSource:0}: Error finding container 334b02302cd976aeb1d6f6acc9f20d8fcfa772e3789b77756dcf5acb9c1540cb: Status 404 returned error can't find the container with id 334b02302cd976aeb1d6f6acc9f20d8fcfa772e3789b77756dcf5acb9c1540cb Jul 02 19:23:30.302406 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.302382 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed6128c8-16a2-4a8a-91a9-faa49567dfcb-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5\" (UID: \"ed6128c8-16a2-4a8a-91a9-faa49567dfcb\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5" Jul 02 19:23:30.302512 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.302437 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed6128c8-16a2-4a8a-91a9-faa49567dfcb-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5\" (UID: \"ed6128c8-16a2-4a8a-91a9-faa49567dfcb\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5" Jul 02 19:23:30.302572 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.302524 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq569\" (UniqueName: \"kubernetes.io/projected/ed6128c8-16a2-4a8a-91a9-faa49567dfcb-kube-api-access-xq569\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5\" (UID: \"ed6128c8-16a2-4a8a-91a9-faa49567dfcb\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5" Jul 02 19:23:30.403792 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.403763 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed6128c8-16a2-4a8a-91a9-faa49567dfcb-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5\" (UID: \"ed6128c8-16a2-4a8a-91a9-faa49567dfcb\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5" Jul 02 19:23:30.403924 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.403811 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq569\" (UniqueName: \"kubernetes.io/projected/ed6128c8-16a2-4a8a-91a9-faa49567dfcb-kube-api-access-xq569\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5\" (UID: \"ed6128c8-16a2-4a8a-91a9-faa49567dfcb\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5" Jul 02 19:23:30.403924 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.403856 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed6128c8-16a2-4a8a-91a9-faa49567dfcb-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5\" (UID: \"ed6128c8-16a2-4a8a-91a9-faa49567dfcb\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5" Jul 02 19:23:30.404197 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.404177 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed6128c8-16a2-4a8a-91a9-faa49567dfcb-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5\" (UID: \"ed6128c8-16a2-4a8a-91a9-faa49567dfcb\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5" Jul 02 19:23:30.404275 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.404206 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed6128c8-16a2-4a8a-91a9-faa49567dfcb-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5\" (UID: \"ed6128c8-16a2-4a8a-91a9-faa49567dfcb\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5" Jul 02 19:23:30.412725 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.412669 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq569\" (UniqueName: \"kubernetes.io/projected/ed6128c8-16a2-4a8a-91a9-faa49567dfcb-kube-api-access-xq569\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5\" (UID: \"ed6128c8-16a2-4a8a-91a9-faa49567dfcb\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5" Jul 02 19:23:30.512645 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.512609 2556 generic.go:356] "Generic (PLEG): container finished" podID="1c48642e-167b-466b-91a4-08490e74cf80" containerID="fdcecb89ff76264796581341e33388852297d34f61eb925dc1e7cd059d8584f1" exitCode=0 Jul 02 19:23:30.512819 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.512697 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq" event={"ID":"1c48642e-167b-466b-91a4-08490e74cf80","Type":"ContainerDied","Data":"fdcecb89ff76264796581341e33388852297d34f61eb925dc1e7cd059d8584f1"} Jul 02 19:23:30.512819 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.512751 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq" event={"ID":"1c48642e-167b-466b-91a4-08490e74cf80","Type":"ContainerStarted","Data":"334b02302cd976aeb1d6f6acc9f20d8fcfa772e3789b77756dcf5acb9c1540cb"} Jul 02 19:23:30.514406 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.514383 2556 generic.go:356] "Generic (PLEG): container finished" podID="edfcfd82-124a-4ff2-be30-a65941f85ce5" containerID="c2f5eb36cea66dde2370adb0caca4098adbb9376e4714278241240f5aefb6984" exitCode=0 Jul 02 19:23:30.514490 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.514428 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65" event={"ID":"edfcfd82-124a-4ff2-be30-a65941f85ce5","Type":"ContainerDied","Data":"c2f5eb36cea66dde2370adb0caca4098adbb9376e4714278241240f5aefb6984"} Jul 02 19:23:30.514490 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.514457 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65" event={"ID":"edfcfd82-124a-4ff2-be30-a65941f85ce5","Type":"ContainerStarted","Data":"b06857f3c786fc40342cd3e98331f50b101a585857817b5e4bae6f43e7a9a4c5"} Jul 02 19:23:30.556813 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.556777 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5" Jul 02 19:23:30.653730 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.653684 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7"] Jul 02 19:23:30.658613 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.658585 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7" Jul 02 19:23:30.661695 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.661661 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7"] Jul 02 19:23:30.694082 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.694058 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5"] Jul 02 19:23:30.696355 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:23:30.696326 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded6128c8_16a2_4a8a_91a9_faa49567dfcb.slice/crio-b5548272d53df70628f06bac43affb308dcb177cdf38c264c9f0168fd4b8e624 WatchSource:0}: Error finding container b5548272d53df70628f06bac43affb308dcb177cdf38c264c9f0168fd4b8e624: Status 404 returned error can't find the container with id b5548272d53df70628f06bac43affb308dcb177cdf38c264c9f0168fd4b8e624 Jul 02 19:23:30.706454 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.706430 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5daaf506-24ce-43da-9389-c59d0d44f1bb-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7\" (UID: \"5daaf506-24ce-43da-9389-c59d0d44f1bb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7" Jul 02 19:23:30.706551 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.706527 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5daaf506-24ce-43da-9389-c59d0d44f1bb-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7\" (UID: \"5daaf506-24ce-43da-9389-c59d0d44f1bb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7" Jul 02 19:23:30.706682 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.706654 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drcq5\" (UniqueName: \"kubernetes.io/projected/5daaf506-24ce-43da-9389-c59d0d44f1bb-kube-api-access-drcq5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7\" (UID: \"5daaf506-24ce-43da-9389-c59d0d44f1bb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7" Jul 02 19:23:30.807215 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.807185 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drcq5\" (UniqueName: \"kubernetes.io/projected/5daaf506-24ce-43da-9389-c59d0d44f1bb-kube-api-access-drcq5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7\" (UID: \"5daaf506-24ce-43da-9389-c59d0d44f1bb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7" Jul 02 19:23:30.807341 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.807225 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5daaf506-24ce-43da-9389-c59d0d44f1bb-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7\" (UID: \"5daaf506-24ce-43da-9389-c59d0d44f1bb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7" Jul 02 19:23:30.807392 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.807342 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5daaf506-24ce-43da-9389-c59d0d44f1bb-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7\" (UID: \"5daaf506-24ce-43da-9389-c59d0d44f1bb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7" Jul 02 19:23:30.807575 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.807557 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5daaf506-24ce-43da-9389-c59d0d44f1bb-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7\" (UID: \"5daaf506-24ce-43da-9389-c59d0d44f1bb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7" Jul 02 19:23:30.807642 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.807629 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5daaf506-24ce-43da-9389-c59d0d44f1bb-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7\" (UID: \"5daaf506-24ce-43da-9389-c59d0d44f1bb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7" Jul 02 19:23:30.816673 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.816646 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drcq5\" (UniqueName: \"kubernetes.io/projected/5daaf506-24ce-43da-9389-c59d0d44f1bb-kube-api-access-drcq5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7\" (UID: \"5daaf506-24ce-43da-9389-c59d0d44f1bb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7" Jul 02 19:23:30.969618 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:30.969583 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7" Jul 02 19:23:31.097585 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:31.097560 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7"] Jul 02 19:23:31.099466 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:23:31.099436 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5daaf506_24ce_43da_9389_c59d0d44f1bb.slice/crio-5ca044bb5c6394ae1c30b3a85c3f367deb309e27da9b331cce741dccf4146f50 WatchSource:0}: Error finding container 5ca044bb5c6394ae1c30b3a85c3f367deb309e27da9b331cce741dccf4146f50: Status 404 returned error can't find the container with id 5ca044bb5c6394ae1c30b3a85c3f367deb309e27da9b331cce741dccf4146f50 Jul 02 19:23:31.522828 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:31.522797 2556 generic.go:356] "Generic (PLEG): container finished" podID="5daaf506-24ce-43da-9389-c59d0d44f1bb" containerID="8221f3fae743fcef2ca40a07e5088eb6423516d8286d6cf1c08115e7b3149784" exitCode=0 Jul 02 19:23:31.523214 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:31.522875 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7" event={"ID":"5daaf506-24ce-43da-9389-c59d0d44f1bb","Type":"ContainerDied","Data":"8221f3fae743fcef2ca40a07e5088eb6423516d8286d6cf1c08115e7b3149784"} Jul 02 19:23:31.523214 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:31.522932 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7" event={"ID":"5daaf506-24ce-43da-9389-c59d0d44f1bb","Type":"ContainerStarted","Data":"5ca044bb5c6394ae1c30b3a85c3f367deb309e27da9b331cce741dccf4146f50"} Jul 02 19:23:31.524581 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:31.524557 2556 generic.go:356] "Generic (PLEG): container finished" podID="ed6128c8-16a2-4a8a-91a9-faa49567dfcb" containerID="1eee38c86fd4f004497465eda3a51eec69b40eeb14beb80e9885f9146873417a" exitCode=0 Jul 02 19:23:31.524715 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:31.524691 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5" event={"ID":"ed6128c8-16a2-4a8a-91a9-faa49567dfcb","Type":"ContainerDied","Data":"1eee38c86fd4f004497465eda3a51eec69b40eeb14beb80e9885f9146873417a"} Jul 02 19:23:31.524784 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:31.524730 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5" event={"ID":"ed6128c8-16a2-4a8a-91a9-faa49567dfcb","Type":"ContainerStarted","Data":"b5548272d53df70628f06bac43affb308dcb177cdf38c264c9f0168fd4b8e624"} Jul 02 19:23:31.527173 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:31.527134 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65" event={"ID":"edfcfd82-124a-4ff2-be30-a65941f85ce5","Type":"ContainerStarted","Data":"7b8ba5dfc0836940c6066bd3fc34249097ec64bafc198d67e316e621c7cdf722"} Jul 02 19:23:32.533861 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:32.533827 2556 generic.go:356] "Generic (PLEG): container finished" podID="1c48642e-167b-466b-91a4-08490e74cf80" containerID="38d2b3d9eec65bfe4733846b0a224fa97a34b5445eef81df64cfff96c5e31d9a" exitCode=0 Jul 02 19:23:32.534322 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:32.533906 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq" event={"ID":"1c48642e-167b-466b-91a4-08490e74cf80","Type":"ContainerDied","Data":"38d2b3d9eec65bfe4733846b0a224fa97a34b5445eef81df64cfff96c5e31d9a"} Jul 02 19:23:32.535931 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:32.535868 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7" event={"ID":"5daaf506-24ce-43da-9389-c59d0d44f1bb","Type":"ContainerStarted","Data":"7edcb9b5962396c5c4870682bd98569e2810cbda43d59204de9f1afee4828b04"} Jul 02 19:23:32.537716 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:32.537695 2556 generic.go:356] "Generic (PLEG): container finished" podID="ed6128c8-16a2-4a8a-91a9-faa49567dfcb" containerID="92e5a3d7a27222861e1fa7006ca7f822504239fc6f110d952b4955fb4f65653c" exitCode=0 Jul 02 19:23:32.537810 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:32.537725 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5" event={"ID":"ed6128c8-16a2-4a8a-91a9-faa49567dfcb","Type":"ContainerDied","Data":"92e5a3d7a27222861e1fa7006ca7f822504239fc6f110d952b4955fb4f65653c"} Jul 02 19:23:32.539795 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:32.539772 2556 generic.go:356] "Generic (PLEG): container finished" podID="edfcfd82-124a-4ff2-be30-a65941f85ce5" containerID="7b8ba5dfc0836940c6066bd3fc34249097ec64bafc198d67e316e621c7cdf722" exitCode=0 Jul 02 19:23:32.539867 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:32.539800 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65" event={"ID":"edfcfd82-124a-4ff2-be30-a65941f85ce5","Type":"ContainerDied","Data":"7b8ba5dfc0836940c6066bd3fc34249097ec64bafc198d67e316e621c7cdf722"} Jul 02 19:23:33.547200 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:33.547165 2556 generic.go:356] "Generic (PLEG): container finished" podID="1c48642e-167b-466b-91a4-08490e74cf80" containerID="ec8ab4388d22f1e7ca464b6213d9d14ac1dce4f9c66ece919b52df70f1eb2417" exitCode=0 Jul 02 19:23:33.547615 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:33.547243 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq" event={"ID":"1c48642e-167b-466b-91a4-08490e74cf80","Type":"ContainerDied","Data":"ec8ab4388d22f1e7ca464b6213d9d14ac1dce4f9c66ece919b52df70f1eb2417"} Jul 02 19:23:33.548976 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:33.548949 2556 generic.go:356] "Generic (PLEG): container finished" podID="5daaf506-24ce-43da-9389-c59d0d44f1bb" containerID="7edcb9b5962396c5c4870682bd98569e2810cbda43d59204de9f1afee4828b04" exitCode=0 Jul 02 19:23:33.549110 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:33.548979 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7" event={"ID":"5daaf506-24ce-43da-9389-c59d0d44f1bb","Type":"ContainerDied","Data":"7edcb9b5962396c5c4870682bd98569e2810cbda43d59204de9f1afee4828b04"} Jul 02 19:23:33.551098 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:33.551081 2556 generic.go:356] "Generic (PLEG): container finished" podID="ed6128c8-16a2-4a8a-91a9-faa49567dfcb" containerID="e067abad0d60071ec9b9e4ffcc4bf82f0e639f17c14f5e743b5a0d6a378caa42" exitCode=0 Jul 02 19:23:33.551220 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:33.551175 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5" event={"ID":"ed6128c8-16a2-4a8a-91a9-faa49567dfcb","Type":"ContainerDied","Data":"e067abad0d60071ec9b9e4ffcc4bf82f0e639f17c14f5e743b5a0d6a378caa42"} Jul 02 19:23:33.553089 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:33.553067 2556 generic.go:356] "Generic (PLEG): container finished" podID="edfcfd82-124a-4ff2-be30-a65941f85ce5" containerID="f01df1dadadf9d7ac34c241c7b6192f9388f95f8ba7c789e35ff6d728467bf18" exitCode=0 Jul 02 19:23:33.553174 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:33.553123 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65" event={"ID":"edfcfd82-124a-4ff2-be30-a65941f85ce5","Type":"ContainerDied","Data":"f01df1dadadf9d7ac34c241c7b6192f9388f95f8ba7c789e35ff6d728467bf18"} Jul 02 19:23:34.559657 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.559623 2556 generic.go:356] "Generic (PLEG): container finished" podID="5daaf506-24ce-43da-9389-c59d0d44f1bb" containerID="80f0c4f2d6b02420ec6f9fcefab578a02517a9225d1b1f331978e80b9f0d49d1" exitCode=0 Jul 02 19:23:34.560065 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.559716 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7" event={"ID":"5daaf506-24ce-43da-9389-c59d0d44f1bb","Type":"ContainerDied","Data":"80f0c4f2d6b02420ec6f9fcefab578a02517a9225d1b1f331978e80b9f0d49d1"} Jul 02 19:23:34.701985 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.701951 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65" Jul 02 19:23:34.729178 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.729132 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq" Jul 02 19:23:34.732069 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.732037 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5" Jul 02 19:23:34.843379 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.843288 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c48642e-167b-466b-91a4-08490e74cf80-util\") pod \"1c48642e-167b-466b-91a4-08490e74cf80\" (UID: \"1c48642e-167b-466b-91a4-08490e74cf80\") " Jul 02 19:23:34.843379 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.843324 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed6128c8-16a2-4a8a-91a9-faa49567dfcb-util\") pod \"ed6128c8-16a2-4a8a-91a9-faa49567dfcb\" (UID: \"ed6128c8-16a2-4a8a-91a9-faa49567dfcb\") " Jul 02 19:23:34.843379 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.843360 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hldpr\" (UniqueName: \"kubernetes.io/projected/1c48642e-167b-466b-91a4-08490e74cf80-kube-api-access-hldpr\") pod \"1c48642e-167b-466b-91a4-08490e74cf80\" (UID: \"1c48642e-167b-466b-91a4-08490e74cf80\") " Jul 02 19:23:34.843667 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.843390 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edfcfd82-124a-4ff2-be30-a65941f85ce5-util\") pod \"edfcfd82-124a-4ff2-be30-a65941f85ce5\" (UID: \"edfcfd82-124a-4ff2-be30-a65941f85ce5\") " Jul 02 19:23:34.843667 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.843407 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed6128c8-16a2-4a8a-91a9-faa49567dfcb-bundle\") pod \"ed6128c8-16a2-4a8a-91a9-faa49567dfcb\" (UID: \"ed6128c8-16a2-4a8a-91a9-faa49567dfcb\") " Jul 02 19:23:34.843667 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.843442 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c48642e-167b-466b-91a4-08490e74cf80-bundle\") pod \"1c48642e-167b-466b-91a4-08490e74cf80\" (UID: \"1c48642e-167b-466b-91a4-08490e74cf80\") " Jul 02 19:23:34.843667 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.843476 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edfcfd82-124a-4ff2-be30-a65941f85ce5-bundle\") pod \"edfcfd82-124a-4ff2-be30-a65941f85ce5\" (UID: \"edfcfd82-124a-4ff2-be30-a65941f85ce5\") " Jul 02 19:23:34.843667 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.843540 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bszfm\" (UniqueName: \"kubernetes.io/projected/edfcfd82-124a-4ff2-be30-a65941f85ce5-kube-api-access-bszfm\") pod \"edfcfd82-124a-4ff2-be30-a65941f85ce5\" (UID: \"edfcfd82-124a-4ff2-be30-a65941f85ce5\") " Jul 02 19:23:34.843667 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.843574 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xq569\" (UniqueName: \"kubernetes.io/projected/ed6128c8-16a2-4a8a-91a9-faa49567dfcb-kube-api-access-xq569\") pod \"ed6128c8-16a2-4a8a-91a9-faa49567dfcb\" (UID: \"ed6128c8-16a2-4a8a-91a9-faa49567dfcb\") " Jul 02 19:23:34.844430 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.844376 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed6128c8-16a2-4a8a-91a9-faa49567dfcb-bundle" (OuterVolumeSpecName: "bundle") pod "ed6128c8-16a2-4a8a-91a9-faa49567dfcb" (UID: "ed6128c8-16a2-4a8a-91a9-faa49567dfcb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:23:34.844593 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.844527 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edfcfd82-124a-4ff2-be30-a65941f85ce5-bundle" (OuterVolumeSpecName: "bundle") pod "edfcfd82-124a-4ff2-be30-a65941f85ce5" (UID: "edfcfd82-124a-4ff2-be30-a65941f85ce5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:23:34.844593 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.844569 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c48642e-167b-466b-91a4-08490e74cf80-bundle" (OuterVolumeSpecName: "bundle") pod "1c48642e-167b-466b-91a4-08490e74cf80" (UID: "1c48642e-167b-466b-91a4-08490e74cf80"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:23:34.846360 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.846337 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c48642e-167b-466b-91a4-08490e74cf80-kube-api-access-hldpr" (OuterVolumeSpecName: "kube-api-access-hldpr") pod "1c48642e-167b-466b-91a4-08490e74cf80" (UID: "1c48642e-167b-466b-91a4-08490e74cf80"). InnerVolumeSpecName "kube-api-access-hldpr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:23:34.846459 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.846361 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edfcfd82-124a-4ff2-be30-a65941f85ce5-kube-api-access-bszfm" (OuterVolumeSpecName: "kube-api-access-bszfm") pod "edfcfd82-124a-4ff2-be30-a65941f85ce5" (UID: "edfcfd82-124a-4ff2-be30-a65941f85ce5"). InnerVolumeSpecName "kube-api-access-bszfm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:23:34.846459 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.846403 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed6128c8-16a2-4a8a-91a9-faa49567dfcb-kube-api-access-xq569" (OuterVolumeSpecName: "kube-api-access-xq569") pod "ed6128c8-16a2-4a8a-91a9-faa49567dfcb" (UID: "ed6128c8-16a2-4a8a-91a9-faa49567dfcb"). InnerVolumeSpecName "kube-api-access-xq569". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:23:34.849739 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.849704 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edfcfd82-124a-4ff2-be30-a65941f85ce5-util" (OuterVolumeSpecName: "util") pod "edfcfd82-124a-4ff2-be30-a65941f85ce5" (UID: "edfcfd82-124a-4ff2-be30-a65941f85ce5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:23:34.849821 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.849768 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed6128c8-16a2-4a8a-91a9-faa49567dfcb-util" (OuterVolumeSpecName: "util") pod "ed6128c8-16a2-4a8a-91a9-faa49567dfcb" (UID: "ed6128c8-16a2-4a8a-91a9-faa49567dfcb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:23:34.850100 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.850081 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c48642e-167b-466b-91a4-08490e74cf80-util" (OuterVolumeSpecName: "util") pod "1c48642e-167b-466b-91a4-08490e74cf80" (UID: "1c48642e-167b-466b-91a4-08490e74cf80"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:23:34.944519 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.944472 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edfcfd82-124a-4ff2-be30-a65941f85ce5-bundle\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:23:34.944519 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.944518 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bszfm\" (UniqueName: \"kubernetes.io/projected/edfcfd82-124a-4ff2-be30-a65941f85ce5-kube-api-access-bszfm\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:23:34.944718 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.944533 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq569\" (UniqueName: \"kubernetes.io/projected/ed6128c8-16a2-4a8a-91a9-faa49567dfcb-kube-api-access-xq569\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:23:34.944718 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.944542 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c48642e-167b-466b-91a4-08490e74cf80-util\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:23:34.944718 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.944552 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed6128c8-16a2-4a8a-91a9-faa49567dfcb-util\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:23:34.944718 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.944559 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hldpr\" (UniqueName: \"kubernetes.io/projected/1c48642e-167b-466b-91a4-08490e74cf80-kube-api-access-hldpr\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:23:34.944718 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.944567 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edfcfd82-124a-4ff2-be30-a65941f85ce5-util\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:23:34.944718 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.944575 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed6128c8-16a2-4a8a-91a9-faa49567dfcb-bundle\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:23:34.944718 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:34.944582 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c48642e-167b-466b-91a4-08490e74cf80-bundle\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:23:35.565949 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:35.565919 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq" Jul 02 19:23:35.566393 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:35.565920 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq" event={"ID":"1c48642e-167b-466b-91a4-08490e74cf80","Type":"ContainerDied","Data":"334b02302cd976aeb1d6f6acc9f20d8fcfa772e3789b77756dcf5acb9c1540cb"} Jul 02 19:23:35.566393 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:35.566052 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="334b02302cd976aeb1d6f6acc9f20d8fcfa772e3789b77756dcf5acb9c1540cb" Jul 02 19:23:35.567863 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:35.567843 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5" Jul 02 19:23:35.567992 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:35.567881 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5" event={"ID":"ed6128c8-16a2-4a8a-91a9-faa49567dfcb","Type":"ContainerDied","Data":"b5548272d53df70628f06bac43affb308dcb177cdf38c264c9f0168fd4b8e624"} Jul 02 19:23:35.567992 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:35.567928 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5548272d53df70628f06bac43affb308dcb177cdf38c264c9f0168fd4b8e624" Jul 02 19:23:35.569843 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:35.569821 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65" event={"ID":"edfcfd82-124a-4ff2-be30-a65941f85ce5","Type":"ContainerDied","Data":"b06857f3c786fc40342cd3e98331f50b101a585857817b5e4bae6f43e7a9a4c5"} Jul 02 19:23:35.569956 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:35.569861 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b06857f3c786fc40342cd3e98331f50b101a585857817b5e4bae6f43e7a9a4c5" Jul 02 19:23:35.569998 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:35.569986 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65" Jul 02 19:23:35.696583 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:35.696559 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7" Jul 02 19:23:35.752296 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:35.752265 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5daaf506-24ce-43da-9389-c59d0d44f1bb-bundle\") pod \"5daaf506-24ce-43da-9389-c59d0d44f1bb\" (UID: \"5daaf506-24ce-43da-9389-c59d0d44f1bb\") " Jul 02 19:23:35.752296 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:35.752301 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drcq5\" (UniqueName: \"kubernetes.io/projected/5daaf506-24ce-43da-9389-c59d0d44f1bb-kube-api-access-drcq5\") pod \"5daaf506-24ce-43da-9389-c59d0d44f1bb\" (UID: \"5daaf506-24ce-43da-9389-c59d0d44f1bb\") " Jul 02 19:23:35.752541 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:35.752347 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5daaf506-24ce-43da-9389-c59d0d44f1bb-util\") pod \"5daaf506-24ce-43da-9389-c59d0d44f1bb\" (UID: \"5daaf506-24ce-43da-9389-c59d0d44f1bb\") " Jul 02 19:23:35.752850 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:35.752822 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5daaf506-24ce-43da-9389-c59d0d44f1bb-bundle" (OuterVolumeSpecName: "bundle") pod "5daaf506-24ce-43da-9389-c59d0d44f1bb" (UID: "5daaf506-24ce-43da-9389-c59d0d44f1bb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:23:35.754439 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:35.754402 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5daaf506-24ce-43da-9389-c59d0d44f1bb-kube-api-access-drcq5" (OuterVolumeSpecName: "kube-api-access-drcq5") pod "5daaf506-24ce-43da-9389-c59d0d44f1bb" (UID: "5daaf506-24ce-43da-9389-c59d0d44f1bb"). InnerVolumeSpecName "kube-api-access-drcq5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:23:35.757887 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:35.757861 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5daaf506-24ce-43da-9389-c59d0d44f1bb-util" (OuterVolumeSpecName: "util") pod "5daaf506-24ce-43da-9389-c59d0d44f1bb" (UID: "5daaf506-24ce-43da-9389-c59d0d44f1bb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:23:35.853796 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:35.853709 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5daaf506-24ce-43da-9389-c59d0d44f1bb-bundle\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:23:35.853796 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:35.853739 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drcq5\" (UniqueName: \"kubernetes.io/projected/5daaf506-24ce-43da-9389-c59d0d44f1bb-kube-api-access-drcq5\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:23:35.853796 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:35.853750 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5daaf506-24ce-43da-9389-c59d0d44f1bb-util\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:23:36.575697 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:36.575667 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7" Jul 02 19:23:36.575697 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:36.575676 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7" event={"ID":"5daaf506-24ce-43da-9389-c59d0d44f1bb","Type":"ContainerDied","Data":"5ca044bb5c6394ae1c30b3a85c3f367deb309e27da9b331cce741dccf4146f50"} Jul 02 19:23:36.576071 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:36.575721 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ca044bb5c6394ae1c30b3a85c3f367deb309e27da9b331cce741dccf4146f50" Jul 02 19:23:38.174637 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:23:38.174602 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5554d9cbbb-nqlrl"] Jul 02 19:24:03.195638 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.195596 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5554d9cbbb-nqlrl" podUID="dc61eeea-b62b-40a8-93e0-5bb748b7c964" containerName="console" containerID="cri-o://48d85ebcf102ad3c62eac9da675bb0c4e21a5a2509be0c40e4176f53dd94301f" gracePeriod=15 Jul 02 19:24:03.441754 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.441729 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:24:03.584591 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.584505 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc61eeea-b62b-40a8-93e0-5bb748b7c964-console-oauth-config\") pod \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " Jul 02 19:24:03.584591 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.584556 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzlwv\" (UniqueName: \"kubernetes.io/projected/dc61eeea-b62b-40a8-93e0-5bb748b7c964-kube-api-access-lzlwv\") pod \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " Jul 02 19:24:03.584591 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.584574 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-trusted-ca-bundle\") pod \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " Jul 02 19:24:03.584867 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.584632 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-oauth-serving-cert\") pod \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " Jul 02 19:24:03.584867 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.584649 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-service-ca\") pod \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " Jul 02 19:24:03.584867 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.584672 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-console-config\") pod \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " Jul 02 19:24:03.584867 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.584695 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc61eeea-b62b-40a8-93e0-5bb748b7c964-console-serving-cert\") pod \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\" (UID: \"dc61eeea-b62b-40a8-93e0-5bb748b7c964\") " Jul 02 19:24:03.585204 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.585171 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "dc61eeea-b62b-40a8-93e0-5bb748b7c964" (UID: "dc61eeea-b62b-40a8-93e0-5bb748b7c964"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:24:03.585204 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.585195 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "dc61eeea-b62b-40a8-93e0-5bb748b7c964" (UID: "dc61eeea-b62b-40a8-93e0-5bb748b7c964"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:24:03.585391 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.585202 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-console-config" (OuterVolumeSpecName: "console-config") pod "dc61eeea-b62b-40a8-93e0-5bb748b7c964" (UID: "dc61eeea-b62b-40a8-93e0-5bb748b7c964"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:24:03.585528 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.585504 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-service-ca" (OuterVolumeSpecName: "service-ca") pod "dc61eeea-b62b-40a8-93e0-5bb748b7c964" (UID: "dc61eeea-b62b-40a8-93e0-5bb748b7c964"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:24:03.586836 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.586807 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc61eeea-b62b-40a8-93e0-5bb748b7c964-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "dc61eeea-b62b-40a8-93e0-5bb748b7c964" (UID: "dc61eeea-b62b-40a8-93e0-5bb748b7c964"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:24:03.586930 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.586862 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc61eeea-b62b-40a8-93e0-5bb748b7c964-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "dc61eeea-b62b-40a8-93e0-5bb748b7c964" (UID: "dc61eeea-b62b-40a8-93e0-5bb748b7c964"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:24:03.586930 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.586893 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc61eeea-b62b-40a8-93e0-5bb748b7c964-kube-api-access-lzlwv" (OuterVolumeSpecName: "kube-api-access-lzlwv") pod "dc61eeea-b62b-40a8-93e0-5bb748b7c964" (UID: "dc61eeea-b62b-40a8-93e0-5bb748b7c964"). InnerVolumeSpecName "kube-api-access-lzlwv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:24:03.685520 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.685478 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzlwv\" (UniqueName: \"kubernetes.io/projected/dc61eeea-b62b-40a8-93e0-5bb748b7c964-kube-api-access-lzlwv\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:24:03.685520 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.685521 2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-trusted-ca-bundle\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:24:03.685735 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.685536 2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-oauth-serving-cert\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:24:03.685735 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.685550 2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-service-ca\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:24:03.685735 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.685564 2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc61eeea-b62b-40a8-93e0-5bb748b7c964-console-config\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:24:03.685735 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.685576 2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc61eeea-b62b-40a8-93e0-5bb748b7c964-console-serving-cert\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:24:03.685735 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.685590 2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc61eeea-b62b-40a8-93e0-5bb748b7c964-console-oauth-config\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:24:03.700351 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.700318 2556 generic.go:356] "Generic (PLEG): container finished" podID="dc61eeea-b62b-40a8-93e0-5bb748b7c964" containerID="48d85ebcf102ad3c62eac9da675bb0c4e21a5a2509be0c40e4176f53dd94301f" exitCode=0 Jul 02 19:24:03.700521 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.700377 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5554d9cbbb-nqlrl" Jul 02 19:24:03.700521 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.700399 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5554d9cbbb-nqlrl" event={"ID":"dc61eeea-b62b-40a8-93e0-5bb748b7c964","Type":"ContainerDied","Data":"48d85ebcf102ad3c62eac9da675bb0c4e21a5a2509be0c40e4176f53dd94301f"} Jul 02 19:24:03.700521 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.700454 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5554d9cbbb-nqlrl" event={"ID":"dc61eeea-b62b-40a8-93e0-5bb748b7c964","Type":"ContainerDied","Data":"3be1e922581f706d5a3fac262044e5fe23b95a1669c3a4b3c39d8a2667b99737"} Jul 02 19:24:03.700521 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.700473 2556 scope.go:117] "RemoveContainer" containerID="48d85ebcf102ad3c62eac9da675bb0c4e21a5a2509be0c40e4176f53dd94301f" Jul 02 19:24:03.709460 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.709438 2556 scope.go:117] "RemoveContainer" containerID="48d85ebcf102ad3c62eac9da675bb0c4e21a5a2509be0c40e4176f53dd94301f" Jul 02 19:24:03.709861 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:24:03.709834 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48d85ebcf102ad3c62eac9da675bb0c4e21a5a2509be0c40e4176f53dd94301f\": container with ID starting with 48d85ebcf102ad3c62eac9da675bb0c4e21a5a2509be0c40e4176f53dd94301f not found: ID does not exist" containerID="48d85ebcf102ad3c62eac9da675bb0c4e21a5a2509be0c40e4176f53dd94301f" Jul 02 19:24:03.709961 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.709871 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48d85ebcf102ad3c62eac9da675bb0c4e21a5a2509be0c40e4176f53dd94301f"} err="failed to get container status \"48d85ebcf102ad3c62eac9da675bb0c4e21a5a2509be0c40e4176f53dd94301f\": rpc error: code = NotFound desc = could not find container \"48d85ebcf102ad3c62eac9da675bb0c4e21a5a2509be0c40e4176f53dd94301f\": container with ID starting with 48d85ebcf102ad3c62eac9da675bb0c4e21a5a2509be0c40e4176f53dd94301f not found: ID does not exist" Jul 02 19:24:03.725684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.725652 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5554d9cbbb-nqlrl"] Jul 02 19:24:03.734411 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.734378 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5554d9cbbb-nqlrl"] Jul 02 19:24:03.977417 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:03.977384 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc61eeea-b62b-40a8-93e0-5bb748b7c964" path="/var/lib/kubelet/pods/dc61eeea-b62b-40a8-93e0-5bb748b7c964/volumes" Jul 02 19:24:57.611502 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611420 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611694 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c48642e-167b-466b-91a4-08490e74cf80" containerName="pull" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611706 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c48642e-167b-466b-91a4-08490e74cf80" containerName="pull" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611714 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edfcfd82-124a-4ff2-be30-a65941f85ce5" containerName="extract" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611720 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="edfcfd82-124a-4ff2-be30-a65941f85ce5" containerName="extract" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611731 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc61eeea-b62b-40a8-93e0-5bb748b7c964" containerName="console" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611740 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc61eeea-b62b-40a8-93e0-5bb748b7c964" containerName="console" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611747 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c48642e-167b-466b-91a4-08490e74cf80" containerName="util" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611752 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c48642e-167b-466b-91a4-08490e74cf80" containerName="util" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611761 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edfcfd82-124a-4ff2-be30-a65941f85ce5" containerName="util" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611766 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="edfcfd82-124a-4ff2-be30-a65941f85ce5" containerName="util" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611773 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c48642e-167b-466b-91a4-08490e74cf80" containerName="extract" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611778 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c48642e-167b-466b-91a4-08490e74cf80" containerName="extract" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611784 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5daaf506-24ce-43da-9389-c59d0d44f1bb" containerName="util" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611789 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5daaf506-24ce-43da-9389-c59d0d44f1bb" containerName="util" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611796 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edfcfd82-124a-4ff2-be30-a65941f85ce5" containerName="pull" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611801 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="edfcfd82-124a-4ff2-be30-a65941f85ce5" containerName="pull" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611815 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed6128c8-16a2-4a8a-91a9-faa49567dfcb" containerName="pull" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611822 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed6128c8-16a2-4a8a-91a9-faa49567dfcb" containerName="pull" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611827 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed6128c8-16a2-4a8a-91a9-faa49567dfcb" containerName="extract" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611831 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed6128c8-16a2-4a8a-91a9-faa49567dfcb" containerName="extract" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611838 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5daaf506-24ce-43da-9389-c59d0d44f1bb" containerName="extract" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611843 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5daaf506-24ce-43da-9389-c59d0d44f1bb" containerName="extract" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611849 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed6128c8-16a2-4a8a-91a9-faa49567dfcb" containerName="util" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611853 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed6128c8-16a2-4a8a-91a9-faa49567dfcb" containerName="util" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611859 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5daaf506-24ce-43da-9389-c59d0d44f1bb" containerName="pull" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611864 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5daaf506-24ce-43da-9389-c59d0d44f1bb" containerName="pull" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611913 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="edfcfd82-124a-4ff2-be30-a65941f85ce5" containerName="extract" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611922 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5daaf506-24ce-43da-9389-c59d0d44f1bb" containerName="extract" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611928 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc61eeea-b62b-40a8-93e0-5bb748b7c964" containerName="console" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611935 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c48642e-167b-466b-91a4-08490e74cf80" containerName="extract" Jul 02 19:24:57.611948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.611941 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed6128c8-16a2-4a8a-91a9-faa49567dfcb" containerName="extract" Jul 02 19:24:57.614683 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.614665 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-c28dd" Jul 02 19:24:57.617138 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.617113 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 02 19:24:57.617264 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.617237 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 19:24:57.617369 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.617333 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 19:24:57.618197 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.618176 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zd8cf\"" Jul 02 19:24:57.623540 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.623512 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:24:57.647467 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.647431 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:24:57.671986 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.671961 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ad960ad7-f120-4c3c-a8fd-330fbf306105-config-file\") pod \"limitador-limitador-54d6d988d5-c28dd\" (UID: \"ad960ad7-f120-4c3c-a8fd-330fbf306105\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-c28dd" Jul 02 19:24:57.672087 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.671991 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chbmz\" (UniqueName: \"kubernetes.io/projected/ad960ad7-f120-4c3c-a8fd-330fbf306105-kube-api-access-chbmz\") pod \"limitador-limitador-54d6d988d5-c28dd\" (UID: \"ad960ad7-f120-4c3c-a8fd-330fbf306105\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-c28dd" Jul 02 19:24:57.773298 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.773259 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ad960ad7-f120-4c3c-a8fd-330fbf306105-config-file\") pod \"limitador-limitador-54d6d988d5-c28dd\" (UID: \"ad960ad7-f120-4c3c-a8fd-330fbf306105\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-c28dd" Jul 02 19:24:57.773298 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.773295 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chbmz\" (UniqueName: \"kubernetes.io/projected/ad960ad7-f120-4c3c-a8fd-330fbf306105-kube-api-access-chbmz\") pod \"limitador-limitador-54d6d988d5-c28dd\" (UID: \"ad960ad7-f120-4c3c-a8fd-330fbf306105\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-c28dd" Jul 02 19:24:57.773966 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.773939 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ad960ad7-f120-4c3c-a8fd-330fbf306105-config-file\") pod \"limitador-limitador-54d6d988d5-c28dd\" (UID: \"ad960ad7-f120-4c3c-a8fd-330fbf306105\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-c28dd" Jul 02 19:24:57.782364 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.782342 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chbmz\" (UniqueName: \"kubernetes.io/projected/ad960ad7-f120-4c3c-a8fd-330fbf306105-kube-api-access-chbmz\") pod \"limitador-limitador-54d6d988d5-c28dd\" (UID: \"ad960ad7-f120-4c3c-a8fd-330fbf306105\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-c28dd" Jul 02 19:24:57.924828 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:57.924794 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-c28dd" Jul 02 19:24:58.054472 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:58.054446 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:24:58.056612 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:24:58.056583 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad960ad7_f120_4c3c_a8fd_330fbf306105.slice/crio-b0794809f575152c0360ae8eab0dfc898b2ac9813d4152143d8b6b8aacf05100 WatchSource:0}: Error finding container b0794809f575152c0360ae8eab0dfc898b2ac9813d4152143d8b6b8aacf05100: Status 404 returned error can't find the container with id b0794809f575152c0360ae8eab0dfc898b2ac9813d4152143d8b6b8aacf05100 Jul 02 19:24:58.948198 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:24:58.948125 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-c28dd" event={"ID":"ad960ad7-f120-4c3c-a8fd-330fbf306105","Type":"ContainerStarted","Data":"b0794809f575152c0360ae8eab0dfc898b2ac9813d4152143d8b6b8aacf05100"} Jul 02 19:25:00.960836 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:25:00.960799 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-c28dd" event={"ID":"ad960ad7-f120-4c3c-a8fd-330fbf306105","Type":"ContainerStarted","Data":"63a65904cf00d1de1c9cf50a7bc53f731343bd1dbdcdd18616c5d2c97769bf22"} Jul 02 19:25:00.961293 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:25:00.960875 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-c28dd" Jul 02 19:25:00.978490 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:25:00.978432 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-c28dd" podStartSLOduration=1.545367122 podStartE2EDuration="3.97841499s" totalImagesPullingTime="2.433047868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:24:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:25:00.977402642 +0000 UTC m=+813.524127662" watchObservedRunningTime="2026-07-02 19:25:00.97841499 +0000 UTC m=+813.525140000" Jul 02 19:25:11.965787 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:25:11.965749 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-c28dd" Jul 02 19:26:10.001649 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:26:10.001464 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:26:43.697388 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:26:43.697353 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:26:49.808978 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:26:49.808942 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:27:00.709340 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:00.709296 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:27:09.396010 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.395968 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j"] Jul 02 19:27:09.399677 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.399656 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.403350 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.403317 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 02 19:27:09.403505 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.403359 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 02 19:27:09.403505 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.403383 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 02 19:27:09.403505 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.403431 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kv7q5\"" Jul 02 19:27:09.410680 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.410651 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j"] Jul 02 19:27:09.517511 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.517470 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3df767f1-f99c-4e57-a70f-294ad26b3f1a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j\" (UID: \"3df767f1-f99c-4e57-a70f-294ad26b3f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.517511 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.517514 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3df767f1-f99c-4e57-a70f-294ad26b3f1a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j\" (UID: \"3df767f1-f99c-4e57-a70f-294ad26b3f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.517763 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.517536 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq8l8\" (UniqueName: \"kubernetes.io/projected/3df767f1-f99c-4e57-a70f-294ad26b3f1a-kube-api-access-sq8l8\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j\" (UID: \"3df767f1-f99c-4e57-a70f-294ad26b3f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.517763 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.517580 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3df767f1-f99c-4e57-a70f-294ad26b3f1a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j\" (UID: \"3df767f1-f99c-4e57-a70f-294ad26b3f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.517763 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.517615 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3df767f1-f99c-4e57-a70f-294ad26b3f1a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j\" (UID: \"3df767f1-f99c-4e57-a70f-294ad26b3f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.517763 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.517643 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3df767f1-f99c-4e57-a70f-294ad26b3f1a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j\" (UID: \"3df767f1-f99c-4e57-a70f-294ad26b3f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.618101 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.618060 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3df767f1-f99c-4e57-a70f-294ad26b3f1a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j\" (UID: \"3df767f1-f99c-4e57-a70f-294ad26b3f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.618304 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.618172 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3df767f1-f99c-4e57-a70f-294ad26b3f1a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j\" (UID: \"3df767f1-f99c-4e57-a70f-294ad26b3f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.618304 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.618209 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3df767f1-f99c-4e57-a70f-294ad26b3f1a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j\" (UID: \"3df767f1-f99c-4e57-a70f-294ad26b3f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.618304 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.618238 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq8l8\" (UniqueName: \"kubernetes.io/projected/3df767f1-f99c-4e57-a70f-294ad26b3f1a-kube-api-access-sq8l8\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j\" (UID: \"3df767f1-f99c-4e57-a70f-294ad26b3f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.618304 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.618263 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3df767f1-f99c-4e57-a70f-294ad26b3f1a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j\" (UID: \"3df767f1-f99c-4e57-a70f-294ad26b3f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.618304 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.618297 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3df767f1-f99c-4e57-a70f-294ad26b3f1a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j\" (UID: \"3df767f1-f99c-4e57-a70f-294ad26b3f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.618594 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.618572 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3df767f1-f99c-4e57-a70f-294ad26b3f1a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j\" (UID: \"3df767f1-f99c-4e57-a70f-294ad26b3f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.618638 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.618603 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3df767f1-f99c-4e57-a70f-294ad26b3f1a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j\" (UID: \"3df767f1-f99c-4e57-a70f-294ad26b3f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.618677 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.618640 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3df767f1-f99c-4e57-a70f-294ad26b3f1a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j\" (UID: \"3df767f1-f99c-4e57-a70f-294ad26b3f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.620516 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.620487 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3df767f1-f99c-4e57-a70f-294ad26b3f1a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j\" (UID: \"3df767f1-f99c-4e57-a70f-294ad26b3f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.620742 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.620725 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3df767f1-f99c-4e57-a70f-294ad26b3f1a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j\" (UID: \"3df767f1-f99c-4e57-a70f-294ad26b3f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.626496 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.626474 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq8l8\" (UniqueName: \"kubernetes.io/projected/3df767f1-f99c-4e57-a70f-294ad26b3f1a-kube-api-access-sq8l8\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j\" (UID: \"3df767f1-f99c-4e57-a70f-294ad26b3f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.712076 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.712021 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:09.867699 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.867673 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j"] Jul 02 19:27:09.869399 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:27:09.869370 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3df767f1_f99c_4e57_a70f_294ad26b3f1a.slice/crio-0c593bfa9bb727a6cd9523863918f5db60f69c9fea396e4d27350b9acb95b0b5 WatchSource:0}: Error finding container 0c593bfa9bb727a6cd9523863918f5db60f69c9fea396e4d27350b9acb95b0b5: Status 404 returned error can't find the container with id 0c593bfa9bb727a6cd9523863918f5db60f69c9fea396e4d27350b9acb95b0b5 Jul 02 19:27:09.870735 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:09.870713 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 19:27:10.537968 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:10.537929 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" event={"ID":"3df767f1-f99c-4e57-a70f-294ad26b3f1a","Type":"ContainerStarted","Data":"0c593bfa9bb727a6cd9523863918f5db60f69c9fea396e4d27350b9acb95b0b5"} Jul 02 19:27:10.812991 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:10.812906 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:27:11.544636 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:11.544596 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" event={"ID":"3df767f1-f99c-4e57-a70f-294ad26b3f1a","Type":"ContainerStarted","Data":"ecb22eee5b6a7bbdfc4d5283b998c8cef7c39dc15b4a47600491c684b757850e"} Jul 02 19:27:11.564753 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:11.564699 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" podStartSLOduration=1.558626071 podStartE2EDuration="2.564683136s" totalImagesPullingTime="1.006057065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:27:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:27:11.563590294 +0000 UTC m=+944.110315325" watchObservedRunningTime="2026-07-02 19:27:11.564683136 +0000 UTC m=+944.111408142" Jul 02 19:27:18.620096 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:18.620057 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:27:19.712359 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:19.712320 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:19.712850 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:19.712391 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:19.725027 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:19.725001 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:20.595315 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:20.595281 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j" Jul 02 19:27:35.509750 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:27:35.509706 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:28:50.103876 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:28:50.103843 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:29:00.902003 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:29:00.901962 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:29:12.296790 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:29:12.296753 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:29:20.713941 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:29:20.713855 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:29:32.398348 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:29:32.398308 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:29:59.904043 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:29:59.904006 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:30:11.951207 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:30:11.951167 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:30:40.502578 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:30:40.502538 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:30:50.296354 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:30:50.296274 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:30:58.406621 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:30:58.406585 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:31:07.806121 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:31:07.806086 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:31:18.809897 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:31:18.809858 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:31:27.909202 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:31:27.909169 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:31:39.401363 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:31:39.401321 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:31:48.714067 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:31:48.714030 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:32:11.601062 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:32:11.601018 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:32:22.005961 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:32:22.005920 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:32:46.100345 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:32:46.100307 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:32:58.303322 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:32:58.303284 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:33:12.407851 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:33:12.407813 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:33:22.504741 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:33:22.504705 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:33:34.205217 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:33:34.205174 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:33:43.398432 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:33:43.398391 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:33:52.301045 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:33:52.300952 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:35:42.801728 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:35:42.801685 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:35:51.600165 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:35:51.600107 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:36:04.301697 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:36:04.301658 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:36:15.906697 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:36:15.906659 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:36:27.196851 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:36:27.196813 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:36:38.010255 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:36:38.010205 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:36:45.909195 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:36:45.909155 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:36:56.802357 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:36:56.802275 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:37:08.703968 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:37:08.703932 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:37:17.107831 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:37:17.107791 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:37:30.200608 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:37:30.200574 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:37:52.406560 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:37:52.406524 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:38:01.401808 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:38:01.401771 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:38:12.203438 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:38:12.203398 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:38:23.594775 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:38:23.594740 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:38:32.210755 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:38:32.210718 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:38:44.802241 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:38:44.802201 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:38:55.102715 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:38:55.102681 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:39:03.806711 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:39:03.806665 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:39:15.109625 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:39:15.109588 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:39:24.107356 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:39:24.107320 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:39:33.693170 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:39:33.693116 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:39:46.810590 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:39:46.810550 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:39:55.609603 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:39:55.609514 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:40:04.998199 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:40:04.998157 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:40:15.603869 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:40:15.603829 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:40:28.040234 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:40:28.040196 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:40:35.800552 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:40:35.800515 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:40:47.497639 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:40:47.497594 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:40:57.799792 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:40:57.799753 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:41:07.905477 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:07.905443 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:41:21.302043 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:21.302007 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:41:30.303002 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:30.302963 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:41:40.799865 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.799827 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n"] Jul 02 19:41:40.803406 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.803386 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.806020 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.805999 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f93f23b9-openshift-default-dockercfg-4plr9\"" Jul 02 19:41:40.827060 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.827031 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n"] Jul 02 19:41:40.882701 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.882666 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-workload-certs\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.882868 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.882710 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-workload-socket\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.882868 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.882733 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72816a99-71e9-444d-81d0-d7bac0740921-istio-podinfo\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.882868 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.882782 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72816a99-71e9-444d-81d0-d7bac0740921-istiod-ca-cert\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.882868 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.882841 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72816a99-71e9-444d-81d0-d7bac0740921-istio-token\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.882868 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.882858 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-istio-data\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.883029 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.882875 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-credential-socket\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.883029 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.882931 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6kvj\" (UniqueName: \"kubernetes.io/projected/72816a99-71e9-444d-81d0-d7bac0740921-kube-api-access-g6kvj\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.883029 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.882964 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-istio-envoy\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.983787 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.983745 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6kvj\" (UniqueName: \"kubernetes.io/projected/72816a99-71e9-444d-81d0-d7bac0740921-kube-api-access-g6kvj\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.983787 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.983792 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-istio-envoy\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.984021 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.983832 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-workload-certs\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.984021 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.983853 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-workload-socket\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.984021 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.983872 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72816a99-71e9-444d-81d0-d7bac0740921-istio-podinfo\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.984021 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.983888 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72816a99-71e9-444d-81d0-d7bac0740921-istiod-ca-cert\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.984021 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.983906 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72816a99-71e9-444d-81d0-d7bac0740921-istio-token\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.984021 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.983923 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-istio-data\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.984021 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.983951 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-credential-socket\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.984478 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.984444 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-workload-socket\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.984611 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.984511 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-credential-socket\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.984611 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.984554 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-istio-data\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.984738 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.984655 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-workload-certs\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.984843 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.984823 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72816a99-71e9-444d-81d0-d7bac0740921-istiod-ca-cert\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.986208 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.986187 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-istio-envoy\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.986479 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.986462 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72816a99-71e9-444d-81d0-d7bac0740921-istio-podinfo\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.992501 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.992476 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72816a99-71e9-444d-81d0-d7bac0740921-istio-token\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:40.992584 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:40.992543 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6kvj\" (UniqueName: \"kubernetes.io/projected/72816a99-71e9-444d-81d0-d7bac0740921-kube-api-access-g6kvj\") pod \"e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:41.113572 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:41.113483 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:41.245817 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:41.245791 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n"] Jul 02 19:41:41.247759 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:41:41.247733 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72816a99_71e9_444d_81d0_d7bac0740921.slice/crio-2c493bce820cfd4bef16ec1ae02b1f9626540caede5f0863370de9d2911c4aef WatchSource:0}: Error finding container 2c493bce820cfd4bef16ec1ae02b1f9626540caede5f0863370de9d2911c4aef: Status 404 returned error can't find the container with id 2c493bce820cfd4bef16ec1ae02b1f9626540caede5f0863370de9d2911c4aef Jul 02 19:41:41.249760 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:41.249739 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 19:41:41.490218 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:41.490171 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" event={"ID":"72816a99-71e9-444d-81d0-d7bac0740921","Type":"ContainerStarted","Data":"2c493bce820cfd4bef16ec1ae02b1f9626540caede5f0863370de9d2911c4aef"} Jul 02 19:41:43.205029 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:43.204978 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:41:43.668787 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:43.668736 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:41:43.668911 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:43.668837 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:41:43.668911 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:43.668879 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:41:44.509355 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:44.509312 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" event={"ID":"72816a99-71e9-444d-81d0-d7bac0740921","Type":"ContainerStarted","Data":"98a093d4e35cf52c66dee2bd0ba5de21be1f1fc445519e60e90b0c45e24e7a59"} Jul 02 19:41:44.530901 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:44.530840 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" podStartSLOduration=2.112348292 podStartE2EDuration="4.530817863s" totalImagesPullingTime="2.418469571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:41:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:41:44.528780384 +0000 UTC m=+1817.075505391" watchObservedRunningTime="2026-07-02 19:41:44.530817863 +0000 UTC m=+1817.077542871" Jul 02 19:41:45.114697 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:45.114654 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:45.119479 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:45.119454 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:45.513609 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:45.513575 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:45.514490 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:45.514472 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:47.238854 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:47.238818 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n"] Jul 02 19:41:47.521802 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:47.521720 2556 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" secret="" err="secret \"e2e-ait-f93f23b9-openshift-default-dockercfg-4plr9\" not found" Jul 02 19:41:48.526012 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:48.525951 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" podUID="72816a99-71e9-444d-81d0-d7bac0740921" containerName="istio-proxy" containerID="cri-o://98a093d4e35cf52c66dee2bd0ba5de21be1f1fc445519e60e90b0c45e24e7a59" gracePeriod=30 Jul 02 19:41:53.763861 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.763820 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:53.892882 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.892798 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6kvj\" (UniqueName: \"kubernetes.io/projected/72816a99-71e9-444d-81d0-d7bac0740921-kube-api-access-g6kvj\") pod \"72816a99-71e9-444d-81d0-d7bac0740921\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " Jul 02 19:41:53.892882 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.892835 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-credential-socket\") pod \"72816a99-71e9-444d-81d0-d7bac0740921\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " Jul 02 19:41:53.892882 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.892874 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72816a99-71e9-444d-81d0-d7bac0740921-istio-podinfo\") pod \"72816a99-71e9-444d-81d0-d7bac0740921\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " Jul 02 19:41:53.893170 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.892897 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-workload-socket\") pod \"72816a99-71e9-444d-81d0-d7bac0740921\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " Jul 02 19:41:53.893170 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.892926 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-istio-envoy\") pod \"72816a99-71e9-444d-81d0-d7bac0740921\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " Jul 02 19:41:53.893170 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.892944 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72816a99-71e9-444d-81d0-d7bac0740921-istio-token\") pod \"72816a99-71e9-444d-81d0-d7bac0740921\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " Jul 02 19:41:53.893170 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.892959 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-istio-data\") pod \"72816a99-71e9-444d-81d0-d7bac0740921\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " Jul 02 19:41:53.893170 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.892990 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-workload-certs\") pod \"72816a99-71e9-444d-81d0-d7bac0740921\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " Jul 02 19:41:53.893170 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.893014 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72816a99-71e9-444d-81d0-d7bac0740921-istiod-ca-cert\") pod \"72816a99-71e9-444d-81d0-d7bac0740921\" (UID: \"72816a99-71e9-444d-81d0-d7bac0740921\") " Jul 02 19:41:53.893483 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.893137 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "72816a99-71e9-444d-81d0-d7bac0740921" (UID: "72816a99-71e9-444d-81d0-d7bac0740921"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:41:53.893483 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.893217 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "72816a99-71e9-444d-81d0-d7bac0740921" (UID: "72816a99-71e9-444d-81d0-d7bac0740921"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:41:53.893483 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.893309 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-credential-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:41:53.893483 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.893328 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-workload-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:41:53.893483 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.893362 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "72816a99-71e9-444d-81d0-d7bac0740921" (UID: "72816a99-71e9-444d-81d0-d7bac0740921"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:41:53.893483 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.893377 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-istio-data" (OuterVolumeSpecName: "istio-data") pod "72816a99-71e9-444d-81d0-d7bac0740921" (UID: "72816a99-71e9-444d-81d0-d7bac0740921"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:41:53.893744 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.893548 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72816a99-71e9-444d-81d0-d7bac0740921-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "72816a99-71e9-444d-81d0-d7bac0740921" (UID: "72816a99-71e9-444d-81d0-d7bac0740921"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:41:53.895214 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.895193 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "72816a99-71e9-444d-81d0-d7bac0740921" (UID: "72816a99-71e9-444d-81d0-d7bac0740921"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:41:53.895347 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.895331 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72816a99-71e9-444d-81d0-d7bac0740921-kube-api-access-g6kvj" (OuterVolumeSpecName: "kube-api-access-g6kvj") pod "72816a99-71e9-444d-81d0-d7bac0740921" (UID: "72816a99-71e9-444d-81d0-d7bac0740921"). InnerVolumeSpecName "kube-api-access-g6kvj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:41:53.895579 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.895554 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/72816a99-71e9-444d-81d0-d7bac0740921-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "72816a99-71e9-444d-81d0-d7bac0740921" (UID: "72816a99-71e9-444d-81d0-d7bac0740921"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:41:53.895673 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.895598 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72816a99-71e9-444d-81d0-d7bac0740921-istio-token" (OuterVolumeSpecName: "istio-token") pod "72816a99-71e9-444d-81d0-d7bac0740921" (UID: "72816a99-71e9-444d-81d0-d7bac0740921"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:41:53.994506 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.994482 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6kvj\" (UniqueName: \"kubernetes.io/projected/72816a99-71e9-444d-81d0-d7bac0740921-kube-api-access-g6kvj\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:41:53.994506 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.994507 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72816a99-71e9-444d-81d0-d7bac0740921-istio-podinfo\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:41:53.994655 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.994516 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-istio-envoy\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:41:53.994655 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.994526 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72816a99-71e9-444d-81d0-d7bac0740921-istio-token\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:41:53.994655 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.994534 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-istio-data\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:41:53.994655 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.994542 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72816a99-71e9-444d-81d0-d7bac0740921-workload-certs\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:41:53.994655 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:53.994550 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72816a99-71e9-444d-81d0-d7bac0740921-istiod-ca-cert\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:41:54.555216 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.555167 2556 generic.go:356] "Generic (PLEG): container finished" podID="72816a99-71e9-444d-81d0-d7bac0740921" containerID="98a093d4e35cf52c66dee2bd0ba5de21be1f1fc445519e60e90b0c45e24e7a59" exitCode=0 Jul 02 19:41:54.555216 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.555211 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" event={"ID":"72816a99-71e9-444d-81d0-d7bac0740921","Type":"ContainerDied","Data":"98a093d4e35cf52c66dee2bd0ba5de21be1f1fc445519e60e90b0c45e24e7a59"} Jul 02 19:41:54.555418 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.555242 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" Jul 02 19:41:54.555418 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.555271 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n" event={"ID":"72816a99-71e9-444d-81d0-d7bac0740921","Type":"ContainerDied","Data":"2c493bce820cfd4bef16ec1ae02b1f9626540caede5f0863370de9d2911c4aef"} Jul 02 19:41:54.555418 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.555300 2556 scope.go:117] "RemoveContainer" containerID="98a093d4e35cf52c66dee2bd0ba5de21be1f1fc445519e60e90b0c45e24e7a59" Jul 02 19:41:54.556941 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.556921 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z"] Jul 02 19:41:54.557329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.557295 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72816a99-71e9-444d-81d0-d7bac0740921" containerName="istio-proxy" Jul 02 19:41:54.557329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.557316 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="72816a99-71e9-444d-81d0-d7bac0740921" containerName="istio-proxy" Jul 02 19:41:54.557483 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.557415 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="72816a99-71e9-444d-81d0-d7bac0740921" containerName="istio-proxy" Jul 02 19:41:54.561852 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.561780 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" Jul 02 19:41:54.565670 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.565648 2556 scope.go:117] "RemoveContainer" containerID="98a093d4e35cf52c66dee2bd0ba5de21be1f1fc445519e60e90b0c45e24e7a59" Jul 02 19:41:54.565983 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:41:54.565960 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98a093d4e35cf52c66dee2bd0ba5de21be1f1fc445519e60e90b0c45e24e7a59\": container with ID starting with 98a093d4e35cf52c66dee2bd0ba5de21be1f1fc445519e60e90b0c45e24e7a59 not found: ID does not exist" containerID="98a093d4e35cf52c66dee2bd0ba5de21be1f1fc445519e60e90b0c45e24e7a59" Jul 02 19:41:54.566073 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.565989 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98a093d4e35cf52c66dee2bd0ba5de21be1f1fc445519e60e90b0c45e24e7a59"} err="failed to get container status \"98a093d4e35cf52c66dee2bd0ba5de21be1f1fc445519e60e90b0c45e24e7a59\": rpc error: code = NotFound desc = could not find container \"98a093d4e35cf52c66dee2bd0ba5de21be1f1fc445519e60e90b0c45e24e7a59\": container with ID starting with 98a093d4e35cf52c66dee2bd0ba5de21be1f1fc445519e60e90b0c45e24e7a59 not found: ID does not exist" Jul 02 19:41:54.566201 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.566181 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-18810d49-serving-cert\"" Jul 02 19:41:54.566688 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.566671 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-wnsfc\"" Jul 02 19:41:54.576968 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.576943 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z"] Jul 02 19:41:54.610684 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.610650 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n"] Jul 02 19:41:54.614414 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.614386 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f93f23b9-openshift-default-5d5f969fbc-6mc7n"] Jul 02 19:41:54.700474 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.700441 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f25238-0293-4b8e-89b4-09e55505bcd6-openshift-service-ca\") pod \"maas-api-e2e-ait-18810d49-6947df4f4b-s856z\" (UID: \"c0f25238-0293-4b8e-89b4-09e55505bcd6\") " pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" Jul 02 19:41:54.700635 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.700488 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0f25238-0293-4b8e-89b4-09e55505bcd6-maas-api-tls\") pod \"maas-api-e2e-ait-18810d49-6947df4f4b-s856z\" (UID: \"c0f25238-0293-4b8e-89b4-09e55505bcd6\") " pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" Jul 02 19:41:54.700635 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.700574 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqlsb\" (UniqueName: \"kubernetes.io/projected/c0f25238-0293-4b8e-89b4-09e55505bcd6-kube-api-access-qqlsb\") pod \"maas-api-e2e-ait-18810d49-6947df4f4b-s856z\" (UID: \"c0f25238-0293-4b8e-89b4-09e55505bcd6\") " pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" Jul 02 19:41:54.801645 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.801604 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f25238-0293-4b8e-89b4-09e55505bcd6-openshift-service-ca\") pod \"maas-api-e2e-ait-18810d49-6947df4f4b-s856z\" (UID: \"c0f25238-0293-4b8e-89b4-09e55505bcd6\") " pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" Jul 02 19:41:54.802047 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.801657 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0f25238-0293-4b8e-89b4-09e55505bcd6-maas-api-tls\") pod \"maas-api-e2e-ait-18810d49-6947df4f4b-s856z\" (UID: \"c0f25238-0293-4b8e-89b4-09e55505bcd6\") " pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" Jul 02 19:41:54.802047 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.801707 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqlsb\" (UniqueName: \"kubernetes.io/projected/c0f25238-0293-4b8e-89b4-09e55505bcd6-kube-api-access-qqlsb\") pod \"maas-api-e2e-ait-18810d49-6947df4f4b-s856z\" (UID: \"c0f25238-0293-4b8e-89b4-09e55505bcd6\") " pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" Jul 02 19:41:54.802047 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:41:54.801829 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-18810d49-serving-cert: secret "maas-api-e2e-ait-18810d49-serving-cert" not found Jul 02 19:41:54.802047 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:41:54.801904 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0f25238-0293-4b8e-89b4-09e55505bcd6-maas-api-tls podName:c0f25238-0293-4b8e-89b4-09e55505bcd6 nodeName:}" failed. No retries permitted until 2026-07-02 19:41:55.301884407 +0000 UTC m=+1827.848609410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c0f25238-0293-4b8e-89b4-09e55505bcd6-maas-api-tls") pod "maas-api-e2e-ait-18810d49-6947df4f4b-s856z" (UID: "c0f25238-0293-4b8e-89b4-09e55505bcd6") : secret "maas-api-e2e-ait-18810d49-serving-cert" not found Jul 02 19:41:54.802284 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.802264 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f25238-0293-4b8e-89b4-09e55505bcd6-openshift-service-ca\") pod \"maas-api-e2e-ait-18810d49-6947df4f4b-s856z\" (UID: \"c0f25238-0293-4b8e-89b4-09e55505bcd6\") " pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" Jul 02 19:41:54.813627 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:54.813586 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqlsb\" (UniqueName: \"kubernetes.io/projected/c0f25238-0293-4b8e-89b4-09e55505bcd6-kube-api-access-qqlsb\") pod \"maas-api-e2e-ait-18810d49-6947df4f4b-s856z\" (UID: \"c0f25238-0293-4b8e-89b4-09e55505bcd6\") " pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" Jul 02 19:41:55.306682 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:55.306647 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0f25238-0293-4b8e-89b4-09e55505bcd6-maas-api-tls\") pod \"maas-api-e2e-ait-18810d49-6947df4f4b-s856z\" (UID: \"c0f25238-0293-4b8e-89b4-09e55505bcd6\") " pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" Jul 02 19:41:55.309044 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:55.309015 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0f25238-0293-4b8e-89b4-09e55505bcd6-maas-api-tls\") pod \"maas-api-e2e-ait-18810d49-6947df4f4b-s856z\" (UID: \"c0f25238-0293-4b8e-89b4-09e55505bcd6\") " pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" Jul 02 19:41:55.474188 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:55.474123 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" Jul 02 19:41:55.608069 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:55.608041 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z"] Jul 02 19:41:55.609900 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:41:55.609874 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0f25238_0293_4b8e_89b4_09e55505bcd6.slice/crio-6d19bab3cdf334d30a1d092f341673f256faf25f0895a3da781ace8f17d3ffa6 WatchSource:0}: Error finding container 6d19bab3cdf334d30a1d092f341673f256faf25f0895a3da781ace8f17d3ffa6: Status 404 returned error can't find the container with id 6d19bab3cdf334d30a1d092f341673f256faf25f0895a3da781ace8f17d3ffa6 Jul 02 19:41:55.978083 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:55.978044 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72816a99-71e9-444d-81d0-d7bac0740921" path="/var/lib/kubelet/pods/72816a99-71e9-444d-81d0-d7bac0740921/volumes" Jul 02 19:41:56.571134 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:56.571092 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" event={"ID":"c0f25238-0293-4b8e-89b4-09e55505bcd6","Type":"ContainerStarted","Data":"6d19bab3cdf334d30a1d092f341673f256faf25f0895a3da781ace8f17d3ffa6"} Jul 02 19:41:58.581952 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:58.581916 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" event={"ID":"c0f25238-0293-4b8e-89b4-09e55505bcd6","Type":"ContainerStarted","Data":"9de307cc855a1c7f436711ef11c1bd8d33093af6b0c577480aec140a53b31efd"} Jul 02 19:41:58.582495 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:58.582054 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" Jul 02 19:41:58.599089 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:58.599038 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" podStartSLOduration=2.463766133 podStartE2EDuration="4.599022625s" totalImagesPullingTime="2.135256492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:41:58.597310841 +0000 UTC m=+1831.144035862" watchObservedRunningTime="2026-07-02 19:41:58.599022625 +0000 UTC m=+1831.145747631" Jul 02 19:41:59.747495 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:41:59.747462 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-18810d49-serving-cert: secret "maas-api-e2e-ait-18810d49-serving-cert" not found Jul 02 19:41:59.747891 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:41:59.747558 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0f25238-0293-4b8e-89b4-09e55505bcd6-maas-api-tls podName:c0f25238-0293-4b8e-89b4-09e55505bcd6 nodeName:}" failed. No retries permitted until 2026-07-02 19:42:00.247535419 +0000 UTC m=+1832.794260428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c0f25238-0293-4b8e-89b4-09e55505bcd6-maas-api-tls") pod "maas-api-e2e-ait-18810d49-6947df4f4b-s856z" (UID: "c0f25238-0293-4b8e-89b4-09e55505bcd6") : secret "maas-api-e2e-ait-18810d49-serving-cert" not found Jul 02 19:41:59.756937 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:41:59.756908 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z"] Jul 02 19:42:00.252214 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:42:00.252184 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-18810d49-serving-cert: secret "maas-api-e2e-ait-18810d49-serving-cert" not found Jul 02 19:42:00.252385 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:42:00.252251 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0f25238-0293-4b8e-89b4-09e55505bcd6-maas-api-tls podName:c0f25238-0293-4b8e-89b4-09e55505bcd6 nodeName:}" failed. No retries permitted until 2026-07-02 19:42:01.252234351 +0000 UTC m=+1833.798959353 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c0f25238-0293-4b8e-89b4-09e55505bcd6-maas-api-tls") pod "maas-api-e2e-ait-18810d49-6947df4f4b-s856z" (UID: "c0f25238-0293-4b8e-89b4-09e55505bcd6") : secret "maas-api-e2e-ait-18810d49-serving-cert" not found Jul 02 19:42:00.590856 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:00.590733 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" podUID="c0f25238-0293-4b8e-89b4-09e55505bcd6" containerName="maas-api" containerID="cri-o://9de307cc855a1c7f436711ef11c1bd8d33093af6b0c577480aec140a53b31efd" gracePeriod=30 Jul 02 19:42:00.829893 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:00.829868 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" Jul 02 19:42:00.958528 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:00.958495 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f25238-0293-4b8e-89b4-09e55505bcd6-openshift-service-ca\") pod \"c0f25238-0293-4b8e-89b4-09e55505bcd6\" (UID: \"c0f25238-0293-4b8e-89b4-09e55505bcd6\") " Jul 02 19:42:00.958693 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:00.958566 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqlsb\" (UniqueName: \"kubernetes.io/projected/c0f25238-0293-4b8e-89b4-09e55505bcd6-kube-api-access-qqlsb\") pod \"c0f25238-0293-4b8e-89b4-09e55505bcd6\" (UID: \"c0f25238-0293-4b8e-89b4-09e55505bcd6\") " Jul 02 19:42:00.958693 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:00.958594 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0f25238-0293-4b8e-89b4-09e55505bcd6-maas-api-tls\") pod \"c0f25238-0293-4b8e-89b4-09e55505bcd6\" (UID: \"c0f25238-0293-4b8e-89b4-09e55505bcd6\") " Jul 02 19:42:00.958950 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:00.958912 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0f25238-0293-4b8e-89b4-09e55505bcd6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c0f25238-0293-4b8e-89b4-09e55505bcd6" (UID: "c0f25238-0293-4b8e-89b4-09e55505bcd6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:42:00.960816 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:00.960781 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0f25238-0293-4b8e-89b4-09e55505bcd6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c0f25238-0293-4b8e-89b4-09e55505bcd6" (UID: "c0f25238-0293-4b8e-89b4-09e55505bcd6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:42:00.960899 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:00.960843 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0f25238-0293-4b8e-89b4-09e55505bcd6-kube-api-access-qqlsb" (OuterVolumeSpecName: "kube-api-access-qqlsb") pod "c0f25238-0293-4b8e-89b4-09e55505bcd6" (UID: "c0f25238-0293-4b8e-89b4-09e55505bcd6"). InnerVolumeSpecName "kube-api-access-qqlsb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:42:01.059417 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:01.059379 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqlsb\" (UniqueName: \"kubernetes.io/projected/c0f25238-0293-4b8e-89b4-09e55505bcd6-kube-api-access-qqlsb\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:42:01.059417 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:01.059410 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0f25238-0293-4b8e-89b4-09e55505bcd6-maas-api-tls\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:42:01.059417 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:01.059419 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f25238-0293-4b8e-89b4-09e55505bcd6-openshift-service-ca\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:42:01.596612 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:01.596574 2556 generic.go:356] "Generic (PLEG): container finished" podID="c0f25238-0293-4b8e-89b4-09e55505bcd6" containerID="9de307cc855a1c7f436711ef11c1bd8d33093af6b0c577480aec140a53b31efd" exitCode=0 Jul 02 19:42:01.596801 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:01.596622 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" Jul 02 19:42:01.596801 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:01.596655 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" event={"ID":"c0f25238-0293-4b8e-89b4-09e55505bcd6","Type":"ContainerDied","Data":"9de307cc855a1c7f436711ef11c1bd8d33093af6b0c577480aec140a53b31efd"} Jul 02 19:42:01.596801 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:01.596706 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z" event={"ID":"c0f25238-0293-4b8e-89b4-09e55505bcd6","Type":"ContainerDied","Data":"6d19bab3cdf334d30a1d092f341673f256faf25f0895a3da781ace8f17d3ffa6"} Jul 02 19:42:01.596801 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:01.596729 2556 scope.go:117] "RemoveContainer" containerID="9de307cc855a1c7f436711ef11c1bd8d33093af6b0c577480aec140a53b31efd" Jul 02 19:42:01.605838 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:01.605817 2556 scope.go:117] "RemoveContainer" containerID="9de307cc855a1c7f436711ef11c1bd8d33093af6b0c577480aec140a53b31efd" Jul 02 19:42:01.606141 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:42:01.606121 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9de307cc855a1c7f436711ef11c1bd8d33093af6b0c577480aec140a53b31efd\": container with ID starting with 9de307cc855a1c7f436711ef11c1bd8d33093af6b0c577480aec140a53b31efd not found: ID does not exist" containerID="9de307cc855a1c7f436711ef11c1bd8d33093af6b0c577480aec140a53b31efd" Jul 02 19:42:01.606224 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:01.606170 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9de307cc855a1c7f436711ef11c1bd8d33093af6b0c577480aec140a53b31efd"} err="failed to get container status \"9de307cc855a1c7f436711ef11c1bd8d33093af6b0c577480aec140a53b31efd\": rpc error: code = NotFound desc = could not find container \"9de307cc855a1c7f436711ef11c1bd8d33093af6b0c577480aec140a53b31efd\": container with ID starting with 9de307cc855a1c7f436711ef11c1bd8d33093af6b0c577480aec140a53b31efd not found: ID does not exist" Jul 02 19:42:01.618443 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:01.618416 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z"] Jul 02 19:42:01.622110 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:01.622087 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-18810d49-6947df4f4b-s856z"] Jul 02 19:42:01.977744 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:01.977702 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0f25238-0293-4b8e-89b4-09e55505bcd6" path="/var/lib/kubelet/pods/c0f25238-0293-4b8e-89b4-09e55505bcd6/volumes" Jul 02 19:42:11.830376 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:11.830335 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n"] Jul 02 19:42:11.830795 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:11.830711 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f25238-0293-4b8e-89b4-09e55505bcd6" containerName="maas-api" Jul 02 19:42:11.830795 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:11.830726 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f25238-0293-4b8e-89b4-09e55505bcd6" containerName="maas-api" Jul 02 19:42:11.830890 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:11.830873 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0f25238-0293-4b8e-89b4-09e55505bcd6" containerName="maas-api" Jul 02 19:42:11.835333 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:11.835313 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:11.839463 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:11.839438 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-dd176bfc-openshift-default-dockercfg-rzfv5\"" Jul 02 19:42:11.846491 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:11.846467 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n"] Jul 02 19:42:11.933301 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:11.933269 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-data\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:11.933469 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:11.933311 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-workload-certs\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:11.933469 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:11.933337 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-podinfo\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:11.933469 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:11.933394 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afc3b4da-6744-418a-b733-c51592bf4cbd-istiod-ca-cert\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:11.933469 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:11.933451 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-envoy\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:11.933469 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:11.933471 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-token\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:11.933469 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:11.933498 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tplr\" (UniqueName: \"kubernetes.io/projected/afc3b4da-6744-418a-b733-c51592bf4cbd-kube-api-access-9tplr\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:11.933804 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:11.933541 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-workload-socket\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:11.933804 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:11.933557 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-credential-socket\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.034773 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.034728 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-token\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.034773 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.034778 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tplr\" (UniqueName: \"kubernetes.io/projected/afc3b4da-6744-418a-b733-c51592bf4cbd-kube-api-access-9tplr\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.035016 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.034809 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-workload-socket\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.035016 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.034836 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-credential-socket\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.035132 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.035061 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-data\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.035132 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.035105 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-workload-certs\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.035265 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.035130 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-podinfo\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.035265 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.035183 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afc3b4da-6744-418a-b733-c51592bf4cbd-istiod-ca-cert\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.035265 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.035245 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-envoy\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.035418 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.035349 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-workload-socket\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.035418 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.035385 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-credential-socket\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.035487 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.035431 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-workload-certs\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.035527 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.035487 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-data\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.035842 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.035819 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afc3b4da-6744-418a-b733-c51592bf4cbd-istiod-ca-cert\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.037374 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.037353 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-envoy\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.037579 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.037561 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-podinfo\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.042632 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.042608 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-token\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.042712 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.042649 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tplr\" (UniqueName: \"kubernetes.io/projected/afc3b4da-6744-418a-b733-c51592bf4cbd-kube-api-access-9tplr\") pod \"e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.147065 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.146981 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:12.278453 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.278410 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n"] Jul 02 19:42:12.280992 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:42:12.280963 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafc3b4da_6744_418a_b733_c51592bf4cbd.slice/crio-46301bb68f4e828f79f82b2874319de070aa4632dbc735798b8fe552f9e7ba3a WatchSource:0}: Error finding container 46301bb68f4e828f79f82b2874319de070aa4632dbc735798b8fe552f9e7ba3a: Status 404 returned error can't find the container with id 46301bb68f4e828f79f82b2874319de070aa4632dbc735798b8fe552f9e7ba3a Jul 02 19:42:12.283097 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.283063 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:42:12.283219 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.283141 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:42:12.283219 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.283206 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:42:12.647063 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.647026 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" event={"ID":"afc3b4da-6744-418a-b733-c51592bf4cbd","Type":"ContainerStarted","Data":"b13e0329a5300682e7eefb65ff9b4ac3d3edc2e42a5d8aeba2caaeb103279226"} Jul 02 19:42:12.647260 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.647091 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" event={"ID":"afc3b4da-6744-418a-b733-c51592bf4cbd","Type":"ContainerStarted","Data":"46301bb68f4e828f79f82b2874319de070aa4632dbc735798b8fe552f9e7ba3a"} Jul 02 19:42:12.667300 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:12.667247 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" podStartSLOduration=1.667229861 podStartE2EDuration="1.667229861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:42:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:42:12.666686696 +0000 UTC m=+1845.213411729" watchObservedRunningTime="2026-07-02 19:42:12.667229861 +0000 UTC m=+1845.213954868" Jul 02 19:42:13.103971 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.103939 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc"] Jul 02 19:42:13.107507 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.107440 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" Jul 02 19:42:13.110086 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.110055 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-dd176bfc-serving-cert\"" Jul 02 19:42:13.110222 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.110167 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-wnsfc\"" Jul 02 19:42:13.120016 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.119991 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc"] Jul 02 19:42:13.144090 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.144055 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed687964-bf99-4b60-bfc2-27ba41dc97fe-maas-api-tls\") pod \"maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc\" (UID: \"ed687964-bf99-4b60-bfc2-27ba41dc97fe\") " pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" Jul 02 19:42:13.144336 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.144130 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr2xc\" (UniqueName: \"kubernetes.io/projected/ed687964-bf99-4b60-bfc2-27ba41dc97fe-kube-api-access-jr2xc\") pod \"maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc\" (UID: \"ed687964-bf99-4b60-bfc2-27ba41dc97fe\") " pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" Jul 02 19:42:13.144336 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.144202 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed687964-bf99-4b60-bfc2-27ba41dc97fe-openshift-service-ca\") pod \"maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc\" (UID: \"ed687964-bf99-4b60-bfc2-27ba41dc97fe\") " pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" Jul 02 19:42:13.148064 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.148031 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:13.153036 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.153011 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:13.245230 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.245202 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed687964-bf99-4b60-bfc2-27ba41dc97fe-maas-api-tls\") pod \"maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc\" (UID: \"ed687964-bf99-4b60-bfc2-27ba41dc97fe\") " pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" Jul 02 19:42:13.245411 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.245264 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr2xc\" (UniqueName: \"kubernetes.io/projected/ed687964-bf99-4b60-bfc2-27ba41dc97fe-kube-api-access-jr2xc\") pod \"maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc\" (UID: \"ed687964-bf99-4b60-bfc2-27ba41dc97fe\") " pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" Jul 02 19:42:13.245411 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.245293 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed687964-bf99-4b60-bfc2-27ba41dc97fe-openshift-service-ca\") pod \"maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc\" (UID: \"ed687964-bf99-4b60-bfc2-27ba41dc97fe\") " pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" Jul 02 19:42:13.245878 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.245849 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed687964-bf99-4b60-bfc2-27ba41dc97fe-openshift-service-ca\") pod \"maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc\" (UID: \"ed687964-bf99-4b60-bfc2-27ba41dc97fe\") " pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" Jul 02 19:42:13.247730 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.247706 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed687964-bf99-4b60-bfc2-27ba41dc97fe-maas-api-tls\") pod \"maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc\" (UID: \"ed687964-bf99-4b60-bfc2-27ba41dc97fe\") " pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" Jul 02 19:42:13.255029 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.255006 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr2xc\" (UniqueName: \"kubernetes.io/projected/ed687964-bf99-4b60-bfc2-27ba41dc97fe-kube-api-access-jr2xc\") pod \"maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc\" (UID: \"ed687964-bf99-4b60-bfc2-27ba41dc97fe\") " pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" Jul 02 19:42:13.418867 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.418819 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" Jul 02 19:42:13.561438 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.561412 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc"] Jul 02 19:42:13.563026 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:42:13.563001 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded687964_bf99_4b60_bfc2_27ba41dc97fe.slice/crio-b5e8f7cb981161f7b1e16d3b6d72ff07374642d40b70f21a6efa8c93f893c118 WatchSource:0}: Error finding container b5e8f7cb981161f7b1e16d3b6d72ff07374642d40b70f21a6efa8c93f893c118: Status 404 returned error can't find the container with id b5e8f7cb981161f7b1e16d3b6d72ff07374642d40b70f21a6efa8c93f893c118 Jul 02 19:42:13.652455 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.652422 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" event={"ID":"ed687964-bf99-4b60-bfc2-27ba41dc97fe","Type":"ContainerStarted","Data":"b5e8f7cb981161f7b1e16d3b6d72ff07374642d40b70f21a6efa8c93f893c118"} Jul 02 19:42:13.652721 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.652705 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:13.653613 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:13.653590 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:14.658850 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:14.658814 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" event={"ID":"ed687964-bf99-4b60-bfc2-27ba41dc97fe","Type":"ContainerStarted","Data":"8d55496e2df5c2945bef350f7a6b7f40c30e5164adf926be24655cce866ddc7c"} Jul 02 19:42:14.659379 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:14.658882 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" Jul 02 19:42:14.680676 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:14.680620 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" podStartSLOduration=1.174059667 podStartE2EDuration="1.68060334s" totalImagesPullingTime="506.543673ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:42:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:42:14.678031672 +0000 UTC m=+1847.224756680" watchObservedRunningTime="2026-07-02 19:42:14.68060334 +0000 UTC m=+1847.227328346" Jul 02 19:42:15.903522 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:15.903484 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:42:17.707432 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:17.707365 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n"] Jul 02 19:42:17.708018 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:17.707985 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" podUID="afc3b4da-6744-418a-b733-c51592bf4cbd" containerName="istio-proxy" containerID="cri-o://b13e0329a5300682e7eefb65ff9b4ac3d3edc2e42a5d8aeba2caaeb103279226" gracePeriod=30 Jul 02 19:42:17.931601 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:17.931568 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc"] Jul 02 19:42:17.931780 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:17.931750 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" podUID="ed687964-bf99-4b60-bfc2-27ba41dc97fe" containerName="maas-api" containerID="cri-o://8d55496e2df5c2945bef350f7a6b7f40c30e5164adf926be24655cce866ddc7c" gracePeriod=30 Jul 02 19:42:18.221916 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:18.221888 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" Jul 02 19:42:18.285485 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:18.285447 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jr2xc\" (UniqueName: \"kubernetes.io/projected/ed687964-bf99-4b60-bfc2-27ba41dc97fe-kube-api-access-jr2xc\") pod \"ed687964-bf99-4b60-bfc2-27ba41dc97fe\" (UID: \"ed687964-bf99-4b60-bfc2-27ba41dc97fe\") " Jul 02 19:42:18.285625 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:18.285496 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed687964-bf99-4b60-bfc2-27ba41dc97fe-openshift-service-ca\") pod \"ed687964-bf99-4b60-bfc2-27ba41dc97fe\" (UID: \"ed687964-bf99-4b60-bfc2-27ba41dc97fe\") " Jul 02 19:42:18.285625 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:18.285515 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed687964-bf99-4b60-bfc2-27ba41dc97fe-maas-api-tls\") pod \"ed687964-bf99-4b60-bfc2-27ba41dc97fe\" (UID: \"ed687964-bf99-4b60-bfc2-27ba41dc97fe\") " Jul 02 19:42:18.285949 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:18.285916 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed687964-bf99-4b60-bfc2-27ba41dc97fe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ed687964-bf99-4b60-bfc2-27ba41dc97fe" (UID: "ed687964-bf99-4b60-bfc2-27ba41dc97fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:42:18.287686 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:18.287667 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed687964-bf99-4b60-bfc2-27ba41dc97fe-kube-api-access-jr2xc" (OuterVolumeSpecName: "kube-api-access-jr2xc") pod "ed687964-bf99-4b60-bfc2-27ba41dc97fe" (UID: "ed687964-bf99-4b60-bfc2-27ba41dc97fe"). InnerVolumeSpecName "kube-api-access-jr2xc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:42:18.287774 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:18.287720 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed687964-bf99-4b60-bfc2-27ba41dc97fe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ed687964-bf99-4b60-bfc2-27ba41dc97fe" (UID: "ed687964-bf99-4b60-bfc2-27ba41dc97fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:42:18.386553 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:18.386458 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jr2xc\" (UniqueName: \"kubernetes.io/projected/ed687964-bf99-4b60-bfc2-27ba41dc97fe-kube-api-access-jr2xc\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:42:18.386553 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:18.386493 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed687964-bf99-4b60-bfc2-27ba41dc97fe-openshift-service-ca\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:42:18.386553 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:18.386502 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed687964-bf99-4b60-bfc2-27ba41dc97fe-maas-api-tls\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:42:18.680430 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:18.680399 2556 generic.go:356] "Generic (PLEG): container finished" podID="ed687964-bf99-4b60-bfc2-27ba41dc97fe" containerID="8d55496e2df5c2945bef350f7a6b7f40c30e5164adf926be24655cce866ddc7c" exitCode=0 Jul 02 19:42:18.680592 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:18.680448 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" Jul 02 19:42:18.680592 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:18.680488 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" event={"ID":"ed687964-bf99-4b60-bfc2-27ba41dc97fe","Type":"ContainerDied","Data":"8d55496e2df5c2945bef350f7a6b7f40c30e5164adf926be24655cce866ddc7c"} Jul 02 19:42:18.680592 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:18.680545 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc" event={"ID":"ed687964-bf99-4b60-bfc2-27ba41dc97fe","Type":"ContainerDied","Data":"b5e8f7cb981161f7b1e16d3b6d72ff07374642d40b70f21a6efa8c93f893c118"} Jul 02 19:42:18.680592 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:18.680573 2556 scope.go:117] "RemoveContainer" containerID="8d55496e2df5c2945bef350f7a6b7f40c30e5164adf926be24655cce866ddc7c" Jul 02 19:42:18.703404 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:18.703366 2556 scope.go:117] "RemoveContainer" containerID="8d55496e2df5c2945bef350f7a6b7f40c30e5164adf926be24655cce866ddc7c" Jul 02 19:42:18.704001 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:42:18.703908 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d55496e2df5c2945bef350f7a6b7f40c30e5164adf926be24655cce866ddc7c\": container with ID starting with 8d55496e2df5c2945bef350f7a6b7f40c30e5164adf926be24655cce866ddc7c not found: ID does not exist" containerID="8d55496e2df5c2945bef350f7a6b7f40c30e5164adf926be24655cce866ddc7c" Jul 02 19:42:18.704001 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:18.703960 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d55496e2df5c2945bef350f7a6b7f40c30e5164adf926be24655cce866ddc7c"} err="failed to get container status \"8d55496e2df5c2945bef350f7a6b7f40c30e5164adf926be24655cce866ddc7c\": rpc error: code = NotFound desc = could not find container \"8d55496e2df5c2945bef350f7a6b7f40c30e5164adf926be24655cce866ddc7c\": container with ID starting with 8d55496e2df5c2945bef350f7a6b7f40c30e5164adf926be24655cce866ddc7c not found: ID does not exist" Jul 02 19:42:18.722250 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:18.722204 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc"] Jul 02 19:42:18.725828 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:18.725799 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-dd176bfc-55c6f9dd8f-qgvcc"] Jul 02 19:42:19.978195 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:19.978164 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed687964-bf99-4b60-bfc2-27ba41dc97fe" path="/var/lib/kubelet/pods/ed687964-bf99-4b60-bfc2-27ba41dc97fe/volumes" Jul 02 19:42:23.708356 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.708321 2556 generic.go:356] "Generic (PLEG): container finished" podID="afc3b4da-6744-418a-b733-c51592bf4cbd" containerID="b13e0329a5300682e7eefb65ff9b4ac3d3edc2e42a5d8aeba2caaeb103279226" exitCode=0 Jul 02 19:42:23.708742 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.708391 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" event={"ID":"afc3b4da-6744-418a-b733-c51592bf4cbd","Type":"ContainerDied","Data":"b13e0329a5300682e7eefb65ff9b4ac3d3edc2e42a5d8aeba2caaeb103279226"} Jul 02 19:42:23.777058 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.776998 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:23.840042 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.840002 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-credential-socket\") pod \"afc3b4da-6744-418a-b733-c51592bf4cbd\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " Jul 02 19:42:23.840042 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.840045 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-workload-certs\") pod \"afc3b4da-6744-418a-b733-c51592bf4cbd\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " Jul 02 19:42:23.840308 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.840066 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-token\") pod \"afc3b4da-6744-418a-b733-c51592bf4cbd\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " Jul 02 19:42:23.840308 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.840102 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-envoy\") pod \"afc3b4da-6744-418a-b733-c51592bf4cbd\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " Jul 02 19:42:23.840308 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.840154 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-workload-socket\") pod \"afc3b4da-6744-418a-b733-c51592bf4cbd\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " Jul 02 19:42:23.840308 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.840192 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tplr\" (UniqueName: \"kubernetes.io/projected/afc3b4da-6744-418a-b733-c51592bf4cbd-kube-api-access-9tplr\") pod \"afc3b4da-6744-418a-b733-c51592bf4cbd\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " Jul 02 19:42:23.840308 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.840210 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-data\") pod \"afc3b4da-6744-418a-b733-c51592bf4cbd\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " Jul 02 19:42:23.840308 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.840242 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-podinfo\") pod \"afc3b4da-6744-418a-b733-c51592bf4cbd\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " Jul 02 19:42:23.840308 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.840266 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afc3b4da-6744-418a-b733-c51592bf4cbd-istiod-ca-cert\") pod \"afc3b4da-6744-418a-b733-c51592bf4cbd\" (UID: \"afc3b4da-6744-418a-b733-c51592bf4cbd\") " Jul 02 19:42:23.840308 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.840298 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "afc3b4da-6744-418a-b733-c51592bf4cbd" (UID: "afc3b4da-6744-418a-b733-c51592bf4cbd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:42:23.840710 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.840385 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "afc3b4da-6744-418a-b733-c51592bf4cbd" (UID: "afc3b4da-6744-418a-b733-c51592bf4cbd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:42:23.840710 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.840523 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-credential-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:42:23.840710 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.840542 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-workload-certs\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:42:23.840710 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.840681 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afc3b4da-6744-418a-b733-c51592bf4cbd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "afc3b4da-6744-418a-b733-c51592bf4cbd" (UID: "afc3b4da-6744-418a-b733-c51592bf4cbd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:42:23.841016 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.840988 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-data" (OuterVolumeSpecName: "istio-data") pod "afc3b4da-6744-418a-b733-c51592bf4cbd" (UID: "afc3b4da-6744-418a-b733-c51592bf4cbd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:42:23.841132 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.841055 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "afc3b4da-6744-418a-b733-c51592bf4cbd" (UID: "afc3b4da-6744-418a-b733-c51592bf4cbd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:42:23.842426 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.842394 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afc3b4da-6744-418a-b733-c51592bf4cbd-kube-api-access-9tplr" (OuterVolumeSpecName: "kube-api-access-9tplr") pod "afc3b4da-6744-418a-b733-c51592bf4cbd" (UID: "afc3b4da-6744-418a-b733-c51592bf4cbd"). InnerVolumeSpecName "kube-api-access-9tplr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:42:23.842547 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.842455 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-token" (OuterVolumeSpecName: "istio-token") pod "afc3b4da-6744-418a-b733-c51592bf4cbd" (UID: "afc3b4da-6744-418a-b733-c51592bf4cbd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:42:23.842716 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.842692 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "afc3b4da-6744-418a-b733-c51592bf4cbd" (UID: "afc3b4da-6744-418a-b733-c51592bf4cbd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:42:23.842797 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.842729 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "afc3b4da-6744-418a-b733-c51592bf4cbd" (UID: "afc3b4da-6744-418a-b733-c51592bf4cbd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:42:23.941530 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.941482 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-podinfo\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:42:23.941530 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.941525 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afc3b4da-6744-418a-b733-c51592bf4cbd-istiod-ca-cert\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:42:23.941530 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.941535 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-token\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:42:23.941530 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.941542 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-envoy\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:42:23.941530 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.941551 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-workload-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:42:23.941831 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.941559 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tplr\" (UniqueName: \"kubernetes.io/projected/afc3b4da-6744-418a-b733-c51592bf4cbd-kube-api-access-9tplr\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:42:23.941831 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:23.941569 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afc3b4da-6744-418a-b733-c51592bf4cbd-istio-data\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:42:24.319421 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:24.319387 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:42:24.713810 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:24.713779 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" Jul 02 19:42:24.714250 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:24.713782 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n" event={"ID":"afc3b4da-6744-418a-b733-c51592bf4cbd","Type":"ContainerDied","Data":"46301bb68f4e828f79f82b2874319de070aa4632dbc735798b8fe552f9e7ba3a"} Jul 02 19:42:24.714250 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:24.713907 2556 scope.go:117] "RemoveContainer" containerID="b13e0329a5300682e7eefb65ff9b4ac3d3edc2e42a5d8aeba2caaeb103279226" Jul 02 19:42:24.737179 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:24.737133 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n"] Jul 02 19:42:24.744937 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:24.744914 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-dd176bfc-openshift-default-9ccc5dff-7nz7n"] Jul 02 19:42:25.977835 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:42:25.977801 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afc3b4da-6744-418a-b733-c51592bf4cbd" path="/var/lib/kubelet/pods/afc3b4da-6744-418a-b733-c51592bf4cbd/volumes" Jul 02 19:43:47.109592 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:47.109557 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:43:55.705527 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:55.705485 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:43:59.036632 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.036597 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t"] Jul 02 19:43:59.036992 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.036870 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed687964-bf99-4b60-bfc2-27ba41dc97fe" containerName="maas-api" Jul 02 19:43:59.036992 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.036881 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed687964-bf99-4b60-bfc2-27ba41dc97fe" containerName="maas-api" Jul 02 19:43:59.036992 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.036893 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afc3b4da-6744-418a-b733-c51592bf4cbd" containerName="istio-proxy" Jul 02 19:43:59.036992 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.036899 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="afc3b4da-6744-418a-b733-c51592bf4cbd" containerName="istio-proxy" Jul 02 19:43:59.036992 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.036955 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="afc3b4da-6744-418a-b733-c51592bf4cbd" containerName="istio-proxy" Jul 02 19:43:59.036992 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.036962 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed687964-bf99-4b60-bfc2-27ba41dc97fe" containerName="maas-api" Jul 02 19:43:59.039824 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.039807 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.042482 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.042456 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1fdbc8cf-openshift-default-dockercfg-g96bv\"" Jul 02 19:43:59.063197 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.063161 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t"] Jul 02 19:43:59.117129 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.117079 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-workload-certs\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.117312 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.117134 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-istio-data\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.117312 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.117220 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-workload-socket\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.117312 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.117288 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-credential-socket\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.117312 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.117308 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83ab9727-950c-4254-a720-df23f00d4377-istio-podinfo\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.117518 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.117337 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-istio-envoy\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.117518 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.117407 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83ab9727-950c-4254-a720-df23f00d4377-istio-token\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.117518 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.117460 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83ab9727-950c-4254-a720-df23f00d4377-istiod-ca-cert\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.117518 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.117490 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rvtz\" (UniqueName: \"kubernetes.io/projected/83ab9727-950c-4254-a720-df23f00d4377-kube-api-access-5rvtz\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.218705 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.218664 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-istio-envoy\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.218705 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.218717 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83ab9727-950c-4254-a720-df23f00d4377-istio-token\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.218968 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.218743 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83ab9727-950c-4254-a720-df23f00d4377-istiod-ca-cert\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.218968 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.218760 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rvtz\" (UniqueName: \"kubernetes.io/projected/83ab9727-950c-4254-a720-df23f00d4377-kube-api-access-5rvtz\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.218968 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.218792 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-workload-certs\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.218968 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.218810 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-istio-data\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.218968 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.218838 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-workload-socket\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.218968 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.218901 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-credential-socket\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.218968 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.218925 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83ab9727-950c-4254-a720-df23f00d4377-istio-podinfo\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.219425 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.219286 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-istio-data\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.219425 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.219351 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-workload-certs\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.219425 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.219393 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-credential-socket\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.219584 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.219430 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-workload-socket\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.219678 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.219659 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83ab9727-950c-4254-a720-df23f00d4377-istiod-ca-cert\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.221003 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.220986 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-istio-envoy\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.221346 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.221330 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83ab9727-950c-4254-a720-df23f00d4377-istio-podinfo\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.227820 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.227797 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rvtz\" (UniqueName: \"kubernetes.io/projected/83ab9727-950c-4254-a720-df23f00d4377-kube-api-access-5rvtz\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.227957 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.227860 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83ab9727-950c-4254-a720-df23f00d4377-istio-token\") pod \"e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.349447 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.349358 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:43:59.487685 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.487656 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t"] Jul 02 19:43:59.489523 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:43:59.489475 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83ab9727_950c_4254_a720_df23f00d4377.slice/crio-7281b3f11a8d3396ae60b441308fc8f544b505e3343e9f8bbc13dc5de7364e3e WatchSource:0}: Error finding container 7281b3f11a8d3396ae60b441308fc8f544b505e3343e9f8bbc13dc5de7364e3e: Status 404 returned error can't find the container with id 7281b3f11a8d3396ae60b441308fc8f544b505e3343e9f8bbc13dc5de7364e3e Jul 02 19:43:59.491709 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.491676 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:43:59.491790 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.491744 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:43:59.491790 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:43:59.491770 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:44:00.144606 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:00.144572 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" event={"ID":"83ab9727-950c-4254-a720-df23f00d4377","Type":"ContainerStarted","Data":"600a6d854d49d5cf94625bf30a24d9005aa1800e88de498ed90ce5c71e3bbda7"} Jul 02 19:44:00.145008 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:00.144626 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" event={"ID":"83ab9727-950c-4254-a720-df23f00d4377","Type":"ContainerStarted","Data":"7281b3f11a8d3396ae60b441308fc8f544b505e3343e9f8bbc13dc5de7364e3e"} Jul 02 19:44:00.166880 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:00.166825 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" podStartSLOduration=1.166809014 podStartE2EDuration="1.166809014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:43:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:44:00.164578145 +0000 UTC m=+1952.711303192" watchObservedRunningTime="2026-07-02 19:44:00.166809014 +0000 UTC m=+1952.713534021" Jul 02 19:44:00.350402 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:00.350365 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:44:00.354986 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:00.354959 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:44:01.150521 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:01.150482 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:44:01.151321 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:01.151299 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:44:02.236646 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:02.236612 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg"] Jul 02 19:44:02.239947 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:02.239923 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" Jul 02 19:44:02.242523 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:02.242494 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-wnsfc\"" Jul 02 19:44:02.242654 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:02.242510 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-1fdbc8cf-serving-cert\"" Jul 02 19:44:02.253513 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:02.253483 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg"] Jul 02 19:44:02.350230 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:02.350188 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86a31d90-5a29-4080-bfa0-b0d53a6a3c24-openshift-service-ca\") pod \"maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg\" (UID: \"86a31d90-5a29-4080-bfa0-b0d53a6a3c24\") " pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" Jul 02 19:44:02.350411 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:02.350234 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knzcf\" (UniqueName: \"kubernetes.io/projected/86a31d90-5a29-4080-bfa0-b0d53a6a3c24-kube-api-access-knzcf\") pod \"maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg\" (UID: \"86a31d90-5a29-4080-bfa0-b0d53a6a3c24\") " pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" Jul 02 19:44:02.350411 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:02.350370 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86a31d90-5a29-4080-bfa0-b0d53a6a3c24-maas-api-tls\") pod \"maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg\" (UID: \"86a31d90-5a29-4080-bfa0-b0d53a6a3c24\") " pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" Jul 02 19:44:02.451083 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:02.451044 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86a31d90-5a29-4080-bfa0-b0d53a6a3c24-maas-api-tls\") pod \"maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg\" (UID: \"86a31d90-5a29-4080-bfa0-b0d53a6a3c24\") " pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" Jul 02 19:44:02.451283 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:02.451105 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86a31d90-5a29-4080-bfa0-b0d53a6a3c24-openshift-service-ca\") pod \"maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg\" (UID: \"86a31d90-5a29-4080-bfa0-b0d53a6a3c24\") " pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" Jul 02 19:44:02.451283 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:02.451126 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knzcf\" (UniqueName: \"kubernetes.io/projected/86a31d90-5a29-4080-bfa0-b0d53a6a3c24-kube-api-access-knzcf\") pod \"maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg\" (UID: \"86a31d90-5a29-4080-bfa0-b0d53a6a3c24\") " pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" Jul 02 19:44:02.451793 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:02.451757 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86a31d90-5a29-4080-bfa0-b0d53a6a3c24-openshift-service-ca\") pod \"maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg\" (UID: \"86a31d90-5a29-4080-bfa0-b0d53a6a3c24\") " pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" Jul 02 19:44:02.453557 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:02.453531 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86a31d90-5a29-4080-bfa0-b0d53a6a3c24-maas-api-tls\") pod \"maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg\" (UID: \"86a31d90-5a29-4080-bfa0-b0d53a6a3c24\") " pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" Jul 02 19:44:02.459909 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:02.459885 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knzcf\" (UniqueName: \"kubernetes.io/projected/86a31d90-5a29-4080-bfa0-b0d53a6a3c24-kube-api-access-knzcf\") pod \"maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg\" (UID: \"86a31d90-5a29-4080-bfa0-b0d53a6a3c24\") " pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" Jul 02 19:44:02.552346 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:02.552258 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" Jul 02 19:44:02.711916 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:02.711889 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg"] Jul 02 19:44:02.713748 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:44:02.713719 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86a31d90_5a29_4080_bfa0_b0d53a6a3c24.slice/crio-591e8852aa42973869e46c5513bc785721ad28b06c50f483e4a157eaebf104e7 WatchSource:0}: Error finding container 591e8852aa42973869e46c5513bc785721ad28b06c50f483e4a157eaebf104e7: Status 404 returned error can't find the container with id 591e8852aa42973869e46c5513bc785721ad28b06c50f483e4a157eaebf104e7 Jul 02 19:44:03.165487 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:03.165451 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" event={"ID":"86a31d90-5a29-4080-bfa0-b0d53a6a3c24","Type":"ContainerStarted","Data":"591e8852aa42973869e46c5513bc785721ad28b06c50f483e4a157eaebf104e7"} Jul 02 19:44:04.171993 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:04.171957 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" event={"ID":"86a31d90-5a29-4080-bfa0-b0d53a6a3c24","Type":"ContainerStarted","Data":"b79c47b4b9e6c556c882212b89af5e007d23f96507cf91a43604f8bcfe1eeb57"} Jul 02 19:44:04.172392 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:04.172117 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" Jul 02 19:44:04.192830 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:04.192770 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" podStartSLOduration=1.692194685 podStartE2EDuration="2.19275388s" totalImagesPullingTime="500.559195ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:44:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:44:04.19094699 +0000 UTC m=+1956.737672023" watchObservedRunningTime="2026-07-02 19:44:04.19275388 +0000 UTC m=+1956.739478887" Jul 02 19:44:07.201598 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:07.201562 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:44:10.182537 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:10.182513 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" Jul 02 19:44:14.360001 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:14.359959 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg"] Jul 02 19:44:14.360432 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:14.360229 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" podUID="86a31d90-5a29-4080-bfa0-b0d53a6a3c24" containerName="maas-api" containerID="cri-o://b79c47b4b9e6c556c882212b89af5e007d23f96507cf91a43604f8bcfe1eeb57" gracePeriod=30 Jul 02 19:44:14.612672 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:14.612615 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" Jul 02 19:44:14.659022 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:14.658992 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knzcf\" (UniqueName: \"kubernetes.io/projected/86a31d90-5a29-4080-bfa0-b0d53a6a3c24-kube-api-access-knzcf\") pod \"86a31d90-5a29-4080-bfa0-b0d53a6a3c24\" (UID: \"86a31d90-5a29-4080-bfa0-b0d53a6a3c24\") " Jul 02 19:44:14.659203 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:14.659070 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86a31d90-5a29-4080-bfa0-b0d53a6a3c24-openshift-service-ca\") pod \"86a31d90-5a29-4080-bfa0-b0d53a6a3c24\" (UID: \"86a31d90-5a29-4080-bfa0-b0d53a6a3c24\") " Jul 02 19:44:14.659203 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:14.659175 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86a31d90-5a29-4080-bfa0-b0d53a6a3c24-maas-api-tls\") pod \"86a31d90-5a29-4080-bfa0-b0d53a6a3c24\" (UID: \"86a31d90-5a29-4080-bfa0-b0d53a6a3c24\") " Jul 02 19:44:14.659553 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:14.659529 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86a31d90-5a29-4080-bfa0-b0d53a6a3c24-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "86a31d90-5a29-4080-bfa0-b0d53a6a3c24" (UID: "86a31d90-5a29-4080-bfa0-b0d53a6a3c24"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:44:14.661137 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:14.661113 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86a31d90-5a29-4080-bfa0-b0d53a6a3c24-kube-api-access-knzcf" (OuterVolumeSpecName: "kube-api-access-knzcf") pod "86a31d90-5a29-4080-bfa0-b0d53a6a3c24" (UID: "86a31d90-5a29-4080-bfa0-b0d53a6a3c24"). InnerVolumeSpecName "kube-api-access-knzcf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:44:14.661244 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:14.661200 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86a31d90-5a29-4080-bfa0-b0d53a6a3c24-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "86a31d90-5a29-4080-bfa0-b0d53a6a3c24" (UID: "86a31d90-5a29-4080-bfa0-b0d53a6a3c24"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:44:14.760323 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:14.760288 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86a31d90-5a29-4080-bfa0-b0d53a6a3c24-openshift-service-ca\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:44:14.760323 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:14.760319 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86a31d90-5a29-4080-bfa0-b0d53a6a3c24-maas-api-tls\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:44:14.760323 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:14.760328 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knzcf\" (UniqueName: \"kubernetes.io/projected/86a31d90-5a29-4080-bfa0-b0d53a6a3c24-kube-api-access-knzcf\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:44:15.237059 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:15.237022 2556 generic.go:356] "Generic (PLEG): container finished" podID="86a31d90-5a29-4080-bfa0-b0d53a6a3c24" containerID="b79c47b4b9e6c556c882212b89af5e007d23f96507cf91a43604f8bcfe1eeb57" exitCode=0 Jul 02 19:44:15.237311 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:15.237069 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" Jul 02 19:44:15.237311 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:15.237117 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" event={"ID":"86a31d90-5a29-4080-bfa0-b0d53a6a3c24","Type":"ContainerDied","Data":"b79c47b4b9e6c556c882212b89af5e007d23f96507cf91a43604f8bcfe1eeb57"} Jul 02 19:44:15.237311 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:15.237200 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg" event={"ID":"86a31d90-5a29-4080-bfa0-b0d53a6a3c24","Type":"ContainerDied","Data":"591e8852aa42973869e46c5513bc785721ad28b06c50f483e4a157eaebf104e7"} Jul 02 19:44:15.237311 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:15.237233 2556 scope.go:117] "RemoveContainer" containerID="b79c47b4b9e6c556c882212b89af5e007d23f96507cf91a43604f8bcfe1eeb57" Jul 02 19:44:15.238854 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:15.238834 2556 generic.go:356] "Generic (PLEG): container finished" podID="83ab9727-950c-4254-a720-df23f00d4377" containerID="600a6d854d49d5cf94625bf30a24d9005aa1800e88de498ed90ce5c71e3bbda7" exitCode=0 Jul 02 19:44:15.238958 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:15.238874 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" event={"ID":"83ab9727-950c-4254-a720-df23f00d4377","Type":"ContainerDied","Data":"600a6d854d49d5cf94625bf30a24d9005aa1800e88de498ed90ce5c71e3bbda7"} Jul 02 19:44:15.239231 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:15.239217 2556 scope.go:117] "RemoveContainer" containerID="600a6d854d49d5cf94625bf30a24d9005aa1800e88de498ed90ce5c71e3bbda7" Jul 02 19:44:15.240190 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:15.240163 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:44:15.240273 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:15.240230 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:44:15.240273 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:15.240267 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:44:15.246113 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:15.246096 2556 scope.go:117] "RemoveContainer" containerID="b79c47b4b9e6c556c882212b89af5e007d23f96507cf91a43604f8bcfe1eeb57" Jul 02 19:44:15.246404 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:44:15.246384 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b79c47b4b9e6c556c882212b89af5e007d23f96507cf91a43604f8bcfe1eeb57\": container with ID starting with b79c47b4b9e6c556c882212b89af5e007d23f96507cf91a43604f8bcfe1eeb57 not found: ID does not exist" containerID="b79c47b4b9e6c556c882212b89af5e007d23f96507cf91a43604f8bcfe1eeb57" Jul 02 19:44:15.246485 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:15.246410 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b79c47b4b9e6c556c882212b89af5e007d23f96507cf91a43604f8bcfe1eeb57"} err="failed to get container status \"b79c47b4b9e6c556c882212b89af5e007d23f96507cf91a43604f8bcfe1eeb57\": rpc error: code = NotFound desc = could not find container \"b79c47b4b9e6c556c882212b89af5e007d23f96507cf91a43604f8bcfe1eeb57\": container with ID starting with b79c47b4b9e6c556c882212b89af5e007d23f96507cf91a43604f8bcfe1eeb57 not found: ID does not exist" Jul 02 19:44:15.276296 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:15.276231 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg"] Jul 02 19:44:15.282107 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:15.282083 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-1fdbc8cf-6df7659ccf-nr2lg"] Jul 02 19:44:15.982707 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:15.982668 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86a31d90-5a29-4080-bfa0-b0d53a6a3c24" path="/var/lib/kubelet/pods/86a31d90-5a29-4080-bfa0-b0d53a6a3c24/volumes" Jul 02 19:44:16.151332 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:16.151279 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:44:16.245792 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:16.245705 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" event={"ID":"83ab9727-950c-4254-a720-df23f00d4377","Type":"ContainerStarted","Data":"d7921ef4cf61268ac92ace22f2e10bcf61d9359981ac8dee392330a26159455e"} Jul 02 19:44:16.349888 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:16.349847 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:44:16.350846 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:16.350820 2556 patch_prober.go:28] interesting pod/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 02 19:44:16.350944 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:16.350882 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" podUID="83ab9727-950c-4254-a720-df23f00d4377" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 02 19:44:17.350080 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:17.350044 2556 patch_prober.go:28] interesting pod/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 02 19:44:17.350517 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:17.350097 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" podUID="83ab9727-950c-4254-a720-df23f00d4377" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 02 19:44:18.197494 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:18.197456 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:44:18.350597 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:18.350555 2556 patch_prober.go:28] interesting pod/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 02 19:44:18.350992 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:18.350613 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" podUID="83ab9727-950c-4254-a720-df23f00d4377" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 02 19:44:19.350247 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:19.350209 2556 patch_prober.go:28] interesting pod/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 02 19:44:19.350462 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:19.350265 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" podUID="83ab9727-950c-4254-a720-df23f00d4377" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 02 19:44:20.354031 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:20.353950 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:44:20.354425 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:20.354341 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:44:20.354932 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:20.354916 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:44:20.639299 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:20.639207 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t"] Jul 02 19:44:21.269923 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:21.269887 2556 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" secret="" err="secret \"e2e-mt-1fdbc8cf-openshift-default-dockercfg-g96bv\" not found" Jul 02 19:44:22.273956 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:22.273896 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" podUID="83ab9727-950c-4254-a720-df23f00d4377" containerName="istio-proxy" containerID="cri-o://d7921ef4cf61268ac92ace22f2e10bcf61d9359981ac8dee392330a26159455e" gracePeriod=30 Jul 02 19:44:27.523497 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.523471 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:44:27.685400 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.685347 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83ab9727-950c-4254-a720-df23f00d4377-istio-podinfo\") pod \"83ab9727-950c-4254-a720-df23f00d4377\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " Jul 02 19:44:27.685400 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.685408 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rvtz\" (UniqueName: \"kubernetes.io/projected/83ab9727-950c-4254-a720-df23f00d4377-kube-api-access-5rvtz\") pod \"83ab9727-950c-4254-a720-df23f00d4377\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " Jul 02 19:44:27.685668 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.685437 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-istio-envoy\") pod \"83ab9727-950c-4254-a720-df23f00d4377\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " Jul 02 19:44:27.685668 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.685470 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-credential-socket\") pod \"83ab9727-950c-4254-a720-df23f00d4377\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " Jul 02 19:44:27.685668 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.685487 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83ab9727-950c-4254-a720-df23f00d4377-istiod-ca-cert\") pod \"83ab9727-950c-4254-a720-df23f00d4377\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " Jul 02 19:44:27.685841 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.685677 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-workload-certs\") pod \"83ab9727-950c-4254-a720-df23f00d4377\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " Jul 02 19:44:27.685841 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.685745 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-workload-socket\") pod \"83ab9727-950c-4254-a720-df23f00d4377\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " Jul 02 19:44:27.685841 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.685777 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83ab9727-950c-4254-a720-df23f00d4377-istio-token\") pod \"83ab9727-950c-4254-a720-df23f00d4377\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " Jul 02 19:44:27.685841 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.685781 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "83ab9727-950c-4254-a720-df23f00d4377" (UID: "83ab9727-950c-4254-a720-df23f00d4377"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:44:27.685841 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.685804 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-istio-data\") pod \"83ab9727-950c-4254-a720-df23f00d4377\" (UID: \"83ab9727-950c-4254-a720-df23f00d4377\") " Jul 02 19:44:27.686083 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.685918 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83ab9727-950c-4254-a720-df23f00d4377-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "83ab9727-950c-4254-a720-df23f00d4377" (UID: "83ab9727-950c-4254-a720-df23f00d4377"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:44:27.686083 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.685996 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "83ab9727-950c-4254-a720-df23f00d4377" (UID: "83ab9727-950c-4254-a720-df23f00d4377"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:44:27.686083 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.685917 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "83ab9727-950c-4254-a720-df23f00d4377" (UID: "83ab9727-950c-4254-a720-df23f00d4377"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:44:27.686284 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.686081 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-credential-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:44:27.686284 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.686099 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83ab9727-950c-4254-a720-df23f00d4377-istiod-ca-cert\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:44:27.686284 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.686114 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-workload-certs\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:44:27.686284 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.686127 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-workload-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:44:27.686949 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.686919 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-istio-data" (OuterVolumeSpecName: "istio-data") pod "83ab9727-950c-4254-a720-df23f00d4377" (UID: "83ab9727-950c-4254-a720-df23f00d4377"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:44:27.687671 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.687644 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83ab9727-950c-4254-a720-df23f00d4377-kube-api-access-5rvtz" (OuterVolumeSpecName: "kube-api-access-5rvtz") pod "83ab9727-950c-4254-a720-df23f00d4377" (UID: "83ab9727-950c-4254-a720-df23f00d4377"). InnerVolumeSpecName "kube-api-access-5rvtz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:44:27.687805 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.687787 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83ab9727-950c-4254-a720-df23f00d4377-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "83ab9727-950c-4254-a720-df23f00d4377" (UID: "83ab9727-950c-4254-a720-df23f00d4377"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:44:27.687805 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.687796 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "83ab9727-950c-4254-a720-df23f00d4377" (UID: "83ab9727-950c-4254-a720-df23f00d4377"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:44:27.687902 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.687889 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83ab9727-950c-4254-a720-df23f00d4377-istio-token" (OuterVolumeSpecName: "istio-token") pod "83ab9727-950c-4254-a720-df23f00d4377" (UID: "83ab9727-950c-4254-a720-df23f00d4377"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:44:27.786516 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.786464 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83ab9727-950c-4254-a720-df23f00d4377-istio-token\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:44:27.786516 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.786508 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-istio-data\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:44:27.786516 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.786517 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83ab9727-950c-4254-a720-df23f00d4377-istio-podinfo\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:44:27.786516 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.786526 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rvtz\" (UniqueName: \"kubernetes.io/projected/83ab9727-950c-4254-a720-df23f00d4377-kube-api-access-5rvtz\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:44:27.786516 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:27.786535 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83ab9727-950c-4254-a720-df23f00d4377-istio-envoy\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:44:28.302033 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:28.301940 2556 generic.go:356] "Generic (PLEG): container finished" podID="83ab9727-950c-4254-a720-df23f00d4377" containerID="d7921ef4cf61268ac92ace22f2e10bcf61d9359981ac8dee392330a26159455e" exitCode=0 Jul 02 19:44:28.302033 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:28.301985 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" event={"ID":"83ab9727-950c-4254-a720-df23f00d4377","Type":"ContainerDied","Data":"d7921ef4cf61268ac92ace22f2e10bcf61d9359981ac8dee392330a26159455e"} Jul 02 19:44:28.302033 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:28.301999 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" Jul 02 19:44:28.302319 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:28.302055 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t" event={"ID":"83ab9727-950c-4254-a720-df23f00d4377","Type":"ContainerDied","Data":"7281b3f11a8d3396ae60b441308fc8f544b505e3343e9f8bbc13dc5de7364e3e"} Jul 02 19:44:28.302319 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:28.302079 2556 scope.go:117] "RemoveContainer" containerID="d7921ef4cf61268ac92ace22f2e10bcf61d9359981ac8dee392330a26159455e" Jul 02 19:44:28.310369 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:28.310351 2556 scope.go:117] "RemoveContainer" containerID="600a6d854d49d5cf94625bf30a24d9005aa1800e88de498ed90ce5c71e3bbda7" Jul 02 19:44:28.317746 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:28.317724 2556 scope.go:117] "RemoveContainer" containerID="d7921ef4cf61268ac92ace22f2e10bcf61d9359981ac8dee392330a26159455e" Jul 02 19:44:28.318013 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:44:28.317993 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7921ef4cf61268ac92ace22f2e10bcf61d9359981ac8dee392330a26159455e\": container with ID starting with d7921ef4cf61268ac92ace22f2e10bcf61d9359981ac8dee392330a26159455e not found: ID does not exist" containerID="d7921ef4cf61268ac92ace22f2e10bcf61d9359981ac8dee392330a26159455e" Jul 02 19:44:28.318083 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:28.318021 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7921ef4cf61268ac92ace22f2e10bcf61d9359981ac8dee392330a26159455e"} err="failed to get container status \"d7921ef4cf61268ac92ace22f2e10bcf61d9359981ac8dee392330a26159455e\": rpc error: code = NotFound desc = could not find container \"d7921ef4cf61268ac92ace22f2e10bcf61d9359981ac8dee392330a26159455e\": container with ID starting with d7921ef4cf61268ac92ace22f2e10bcf61d9359981ac8dee392330a26159455e not found: ID does not exist" Jul 02 19:44:28.318083 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:28.318039 2556 scope.go:117] "RemoveContainer" containerID="600a6d854d49d5cf94625bf30a24d9005aa1800e88de498ed90ce5c71e3bbda7" Jul 02 19:44:28.318349 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:44:28.318334 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"600a6d854d49d5cf94625bf30a24d9005aa1800e88de498ed90ce5c71e3bbda7\": container with ID starting with 600a6d854d49d5cf94625bf30a24d9005aa1800e88de498ed90ce5c71e3bbda7 not found: ID does not exist" containerID="600a6d854d49d5cf94625bf30a24d9005aa1800e88de498ed90ce5c71e3bbda7" Jul 02 19:44:28.318403 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:28.318353 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"600a6d854d49d5cf94625bf30a24d9005aa1800e88de498ed90ce5c71e3bbda7"} err="failed to get container status \"600a6d854d49d5cf94625bf30a24d9005aa1800e88de498ed90ce5c71e3bbda7\": rpc error: code = NotFound desc = could not find container \"600a6d854d49d5cf94625bf30a24d9005aa1800e88de498ed90ce5c71e3bbda7\": container with ID starting with 600a6d854d49d5cf94625bf30a24d9005aa1800e88de498ed90ce5c71e3bbda7 not found: ID does not exist" Jul 02 19:44:28.321497 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:28.321477 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t"] Jul 02 19:44:28.325516 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:28.325495 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1fdbc8cf-openshift-default-5744fdc9f5-lg22t"] Jul 02 19:44:29.977504 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:29.977467 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83ab9727-950c-4254-a720-df23f00d4377" path="/var/lib/kubelet/pods/83ab9727-950c-4254-a720-df23f00d4377/volumes" Jul 02 19:44:39.502828 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.502789 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9"] Jul 02 19:44:39.503212 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.503059 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83ab9727-950c-4254-a720-df23f00d4377" containerName="istio-proxy" Jul 02 19:44:39.503212 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.503070 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="83ab9727-950c-4254-a720-df23f00d4377" containerName="istio-proxy" Jul 02 19:44:39.503212 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.503085 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86a31d90-5a29-4080-bfa0-b0d53a6a3c24" containerName="maas-api" Jul 02 19:44:39.503212 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.503091 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="86a31d90-5a29-4080-bfa0-b0d53a6a3c24" containerName="maas-api" Jul 02 19:44:39.503212 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.503166 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="83ab9727-950c-4254-a720-df23f00d4377" containerName="istio-proxy" Jul 02 19:44:39.503212 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.503176 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="86a31d90-5a29-4080-bfa0-b0d53a6a3c24" containerName="maas-api" Jul 02 19:44:39.505339 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.505315 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.508087 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.508065 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-363f6558-openshift-default-dockercfg-5f75h\"" Jul 02 19:44:39.519735 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.519710 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9"] Jul 02 19:44:39.681005 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.680957 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istiod-ca-cert\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.681266 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.681064 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-token\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.681266 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.681114 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-envoy\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.681266 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.681191 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwcz7\" (UniqueName: \"kubernetes.io/projected/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-kube-api-access-jwcz7\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.681266 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.681224 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-workload-socket\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.681266 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.681246 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-workload-certs\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.681540 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.681293 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-credential-socket\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.681540 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.681347 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-data\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.681540 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.681371 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-podinfo\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.782187 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.782075 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwcz7\" (UniqueName: \"kubernetes.io/projected/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-kube-api-access-jwcz7\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.782187 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.782119 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-workload-socket\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.782187 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.782136 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-workload-certs\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.782456 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.782190 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-credential-socket\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.782456 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.782250 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-data\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.782456 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.782286 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-podinfo\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.782456 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.782318 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istiod-ca-cert\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.782667 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.782500 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-token\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.782667 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.782544 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-envoy\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.782667 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.782617 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-workload-socket\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.782667 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.782652 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-workload-certs\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.782851 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.782716 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-data\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.782851 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.782757 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-credential-socket\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.783089 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.783073 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istiod-ca-cert\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.784688 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.784669 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-envoy\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.785026 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.785004 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-podinfo\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.790642 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.790619 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-token\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.790746 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.790733 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwcz7\" (UniqueName: \"kubernetes.io/projected/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-kube-api-access-jwcz7\") pod \"e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.814939 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.814894 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:39.944398 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.944374 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9"] Jul 02 19:44:39.946211 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:44:39.946185 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda62be5bb_d2fb_4fcc_9fdd_f55338b6b442.slice/crio-aae3ddbe1dc325d3708079409305306d6a4caa14d350ea598606906c9b8ab814 WatchSource:0}: Error finding container aae3ddbe1dc325d3708079409305306d6a4caa14d350ea598606906c9b8ab814: Status 404 returned error can't find the container with id aae3ddbe1dc325d3708079409305306d6a4caa14d350ea598606906c9b8ab814 Jul 02 19:44:39.948321 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.948287 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:44:39.948416 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.948353 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:44:39.948416 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:39.948380 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:44:40.359267 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:40.359228 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" event={"ID":"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442","Type":"ContainerStarted","Data":"33c42a52b8595f61e28c93a5a5f79bb65f52c7b77edaeebe6f4c659299fe3fc7"} Jul 02 19:44:40.359469 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:40.359281 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" event={"ID":"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442","Type":"ContainerStarted","Data":"aae3ddbe1dc325d3708079409305306d6a4caa14d350ea598606906c9b8ab814"} Jul 02 19:44:40.378731 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:40.378678 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" podStartSLOduration=1.378664025 podStartE2EDuration="1.378664025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:44:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:44:40.377493583 +0000 UTC m=+1992.924218626" watchObservedRunningTime="2026-07-02 19:44:40.378664025 +0000 UTC m=+1992.925389031" Jul 02 19:44:40.815277 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:40.815242 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:41.820439 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:41.820410 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:42.369653 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.369621 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:42.370690 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.370669 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:42.430859 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.430825 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6"] Jul 02 19:44:42.431219 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.431198 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83ab9727-950c-4254-a720-df23f00d4377" containerName="istio-proxy" Jul 02 19:44:42.431219 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.431216 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="83ab9727-950c-4254-a720-df23f00d4377" containerName="istio-proxy" Jul 02 19:44:42.431385 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.431272 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="83ab9727-950c-4254-a720-df23f00d4377" containerName="istio-proxy" Jul 02 19:44:42.433319 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.433298 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" Jul 02 19:44:42.436039 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.436014 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-363f6558-serving-cert\"" Jul 02 19:44:42.436632 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.436610 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-wnsfc\"" Jul 02 19:44:42.445116 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.445082 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6"] Jul 02 19:44:42.603695 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.603661 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b577c01d-9977-4de4-a9c5-b418e5475750-openshift-service-ca\") pod \"maas-api-e2e-mt-363f6558-559f664cd-dqpb6\" (UID: \"b577c01d-9977-4de4-a9c5-b418e5475750\") " pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" Jul 02 19:44:42.603695 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.603699 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lghg\" (UniqueName: \"kubernetes.io/projected/b577c01d-9977-4de4-a9c5-b418e5475750-kube-api-access-4lghg\") pod \"maas-api-e2e-mt-363f6558-559f664cd-dqpb6\" (UID: \"b577c01d-9977-4de4-a9c5-b418e5475750\") " pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" Jul 02 19:44:42.603903 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.603720 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b577c01d-9977-4de4-a9c5-b418e5475750-maas-api-tls\") pod \"maas-api-e2e-mt-363f6558-559f664cd-dqpb6\" (UID: \"b577c01d-9977-4de4-a9c5-b418e5475750\") " pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" Jul 02 19:44:42.704977 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.704945 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b577c01d-9977-4de4-a9c5-b418e5475750-maas-api-tls\") pod \"maas-api-e2e-mt-363f6558-559f664cd-dqpb6\" (UID: \"b577c01d-9977-4de4-a9c5-b418e5475750\") " pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" Jul 02 19:44:42.705204 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.705028 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b577c01d-9977-4de4-a9c5-b418e5475750-openshift-service-ca\") pod \"maas-api-e2e-mt-363f6558-559f664cd-dqpb6\" (UID: \"b577c01d-9977-4de4-a9c5-b418e5475750\") " pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" Jul 02 19:44:42.705204 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.705054 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lghg\" (UniqueName: \"kubernetes.io/projected/b577c01d-9977-4de4-a9c5-b418e5475750-kube-api-access-4lghg\") pod \"maas-api-e2e-mt-363f6558-559f664cd-dqpb6\" (UID: \"b577c01d-9977-4de4-a9c5-b418e5475750\") " pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" Jul 02 19:44:42.705699 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.705681 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b577c01d-9977-4de4-a9c5-b418e5475750-openshift-service-ca\") pod \"maas-api-e2e-mt-363f6558-559f664cd-dqpb6\" (UID: \"b577c01d-9977-4de4-a9c5-b418e5475750\") " pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" Jul 02 19:44:42.707471 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.707452 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b577c01d-9977-4de4-a9c5-b418e5475750-maas-api-tls\") pod \"maas-api-e2e-mt-363f6558-559f664cd-dqpb6\" (UID: \"b577c01d-9977-4de4-a9c5-b418e5475750\") " pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" Jul 02 19:44:42.713843 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.713813 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lghg\" (UniqueName: \"kubernetes.io/projected/b577c01d-9977-4de4-a9c5-b418e5475750-kube-api-access-4lghg\") pod \"maas-api-e2e-mt-363f6558-559f664cd-dqpb6\" (UID: \"b577c01d-9977-4de4-a9c5-b418e5475750\") " pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" Jul 02 19:44:42.745904 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.745867 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" Jul 02 19:44:42.876862 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:42.876836 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6"] Jul 02 19:44:42.879379 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:44:42.879352 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb577c01d_9977_4de4_a9c5_b418e5475750.slice/crio-be497839062c16ed5ac1b711c8c7999b8d10cb3fc14befcb9dd3075a2adaa3ba WatchSource:0}: Error finding container be497839062c16ed5ac1b711c8c7999b8d10cb3fc14befcb9dd3075a2adaa3ba: Status 404 returned error can't find the container with id be497839062c16ed5ac1b711c8c7999b8d10cb3fc14befcb9dd3075a2adaa3ba Jul 02 19:44:43.375201 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:43.375135 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" event={"ID":"b577c01d-9977-4de4-a9c5-b418e5475750","Type":"ContainerStarted","Data":"a04da280585b9f256097158cea68223a041fefe532fa6546dec30cac6fb0475a"} Jul 02 19:44:43.375384 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:43.375365 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" Jul 02 19:44:43.375426 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:43.375402 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" event={"ID":"b577c01d-9977-4de4-a9c5-b418e5475750","Type":"ContainerStarted","Data":"be497839062c16ed5ac1b711c8c7999b8d10cb3fc14befcb9dd3075a2adaa3ba"} Jul 02 19:44:43.393309 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:43.393264 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" podStartSLOduration=1.050974163 podStartE2EDuration="1.393248032s" totalImagesPullingTime="342.273869ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:44:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:44:43.391792691 +0000 UTC m=+1995.938517700" watchObservedRunningTime="2026-07-02 19:44:43.393248032 +0000 UTC m=+1995.939973038" Jul 02 19:44:48.500635 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:48.500599 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:44:49.384943 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:49.384916 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" Jul 02 19:44:56.441058 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:56.441025 2556 generic.go:356] "Generic (PLEG): container finished" podID="a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" containerID="33c42a52b8595f61e28c93a5a5f79bb65f52c7b77edaeebe6f4c659299fe3fc7" exitCode=0 Jul 02 19:44:56.441475 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:56.441069 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" event={"ID":"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442","Type":"ContainerDied","Data":"33c42a52b8595f61e28c93a5a5f79bb65f52c7b77edaeebe6f4c659299fe3fc7"} Jul 02 19:44:56.441518 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:56.441473 2556 scope.go:117] "RemoveContainer" containerID="33c42a52b8595f61e28c93a5a5f79bb65f52c7b77edaeebe6f4c659299fe3fc7" Jul 02 19:44:56.442463 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:56.442432 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:44:56.442528 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:56.442503 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:44:56.442565 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:56.442537 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:44:57.370493 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:57.370455 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:57.447742 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:57.447700 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" event={"ID":"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442","Type":"ContainerStarted","Data":"389b3e9a37bb08eaa1f3eea97fe39fbda2a82cfb5b7722764243606933c8393d"} Jul 02 19:44:57.816063 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:57.816027 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:44:57.817200 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:57.817172 2556 patch_prober.go:28] interesting pod/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 02 19:44:57.817323 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:57.817220 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" podUID="a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 02 19:44:58.816121 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:58.816076 2556 patch_prober.go:28] interesting pod/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 02 19:44:58.816648 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:58.816132 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" podUID="a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 02 19:44:59.702033 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:59.701989 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:44:59.816064 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:59.816013 2556 patch_prober.go:28] interesting pod/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 02 19:44:59.816285 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:44:59.816075 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" podUID="a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 02 19:45:00.816045 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:00.816010 2556 patch_prober.go:28] interesting pod/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 02 19:45:00.816255 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:00.816063 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" podUID="a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 02 19:45:01.815803 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:01.815753 2556 patch_prober.go:28] interesting pod/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 02 19:45:01.816225 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:01.815831 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" podUID="a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 02 19:45:02.820092 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:02.820059 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:45:02.820512 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:02.820388 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:45:02.821048 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:02.821030 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:45:12.006713 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:12.006676 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:45:14.304925 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:14.304881 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6"] Jul 02 19:45:14.305437 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:14.305163 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" podUID="b577c01d-9977-4de4-a9c5-b418e5475750" containerName="maas-api" containerID="cri-o://a04da280585b9f256097158cea68223a041fefe532fa6546dec30cac6fb0475a" gracePeriod=30 Jul 02 19:45:14.380319 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:14.380274 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" podUID="b577c01d-9977-4de4-a9c5-b418e5475750" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.37:8443/health\": dial tcp 10.132.0.37:8443: connect: connection refused" Jul 02 19:45:14.536843 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:14.536812 2556 generic.go:356] "Generic (PLEG): container finished" podID="b577c01d-9977-4de4-a9c5-b418e5475750" containerID="a04da280585b9f256097158cea68223a041fefe532fa6546dec30cac6fb0475a" exitCode=0 Jul 02 19:45:14.537003 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:14.536886 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" event={"ID":"b577c01d-9977-4de4-a9c5-b418e5475750","Type":"ContainerDied","Data":"a04da280585b9f256097158cea68223a041fefe532fa6546dec30cac6fb0475a"} Jul 02 19:45:14.555790 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:14.555735 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" Jul 02 19:45:14.665793 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:14.665758 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b577c01d-9977-4de4-a9c5-b418e5475750-maas-api-tls\") pod \"b577c01d-9977-4de4-a9c5-b418e5475750\" (UID: \"b577c01d-9977-4de4-a9c5-b418e5475750\") " Jul 02 19:45:14.665970 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:14.665804 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lghg\" (UniqueName: \"kubernetes.io/projected/b577c01d-9977-4de4-a9c5-b418e5475750-kube-api-access-4lghg\") pod \"b577c01d-9977-4de4-a9c5-b418e5475750\" (UID: \"b577c01d-9977-4de4-a9c5-b418e5475750\") " Jul 02 19:45:14.665970 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:14.665899 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b577c01d-9977-4de4-a9c5-b418e5475750-openshift-service-ca\") pod \"b577c01d-9977-4de4-a9c5-b418e5475750\" (UID: \"b577c01d-9977-4de4-a9c5-b418e5475750\") " Jul 02 19:45:14.666326 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:14.666299 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b577c01d-9977-4de4-a9c5-b418e5475750-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b577c01d-9977-4de4-a9c5-b418e5475750" (UID: "b577c01d-9977-4de4-a9c5-b418e5475750"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:45:14.667955 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:14.667933 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b577c01d-9977-4de4-a9c5-b418e5475750-kube-api-access-4lghg" (OuterVolumeSpecName: "kube-api-access-4lghg") pod "b577c01d-9977-4de4-a9c5-b418e5475750" (UID: "b577c01d-9977-4de4-a9c5-b418e5475750"). InnerVolumeSpecName "kube-api-access-4lghg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:45:14.667955 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:14.667933 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b577c01d-9977-4de4-a9c5-b418e5475750-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b577c01d-9977-4de4-a9c5-b418e5475750" (UID: "b577c01d-9977-4de4-a9c5-b418e5475750"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:45:14.767276 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:14.767238 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b577c01d-9977-4de4-a9c5-b418e5475750-openshift-service-ca\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:45:14.767276 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:14.767268 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b577c01d-9977-4de4-a9c5-b418e5475750-maas-api-tls\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:45:14.767276 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:14.767281 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lghg\" (UniqueName: \"kubernetes.io/projected/b577c01d-9977-4de4-a9c5-b418e5475750-kube-api-access-4lghg\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:45:15.543442 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:15.543409 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" Jul 02 19:45:15.544038 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:15.543444 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6" event={"ID":"b577c01d-9977-4de4-a9c5-b418e5475750","Type":"ContainerDied","Data":"be497839062c16ed5ac1b711c8c7999b8d10cb3fc14befcb9dd3075a2adaa3ba"} Jul 02 19:45:15.544038 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:15.543508 2556 scope.go:117] "RemoveContainer" containerID="a04da280585b9f256097158cea68223a041fefe532fa6546dec30cac6fb0475a" Jul 02 19:45:15.564445 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:15.564418 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6"] Jul 02 19:45:15.568581 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:15.568561 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-363f6558-559f664cd-dqpb6"] Jul 02 19:45:15.977965 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:15.977933 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b577c01d-9977-4de4-a9c5-b418e5475750" path="/var/lib/kubelet/pods/b577c01d-9977-4de4-a9c5-b418e5475750/volumes" Jul 02 19:45:24.408883 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:45:24.408844 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:46:04.606942 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:04.606854 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:46:08.074620 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:08.074587 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9"] Jul 02 19:46:08.075109 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:08.074803 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" podUID="a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" containerName="istio-proxy" containerID="cri-o://389b3e9a37bb08eaa1f3eea97fe39fbda2a82cfb5b7722764243606933c8393d" gracePeriod=30 Jul 02 19:46:13.320929 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.320904 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:46:13.347188 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.347102 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-workload-certs\") pod \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " Jul 02 19:46:13.347188 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.347136 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-data\") pod \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " Jul 02 19:46:13.347188 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.347180 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-token\") pod \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " Jul 02 19:46:13.347446 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.347206 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwcz7\" (UniqueName: \"kubernetes.io/projected/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-kube-api-access-jwcz7\") pod \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " Jul 02 19:46:13.347446 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.347232 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-podinfo\") pod \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " Jul 02 19:46:13.347446 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.347249 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-envoy\") pod \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " Jul 02 19:46:13.347446 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.347415 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" (UID: "a62be5bb-d2fb-4fcc-9fdd-f55338b6b442"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:46:13.347660 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.347467 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-credential-socket\") pod \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " Jul 02 19:46:13.347660 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.347526 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istiod-ca-cert\") pod \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " Jul 02 19:46:13.347660 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.347557 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-workload-socket\") pod \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\" (UID: \"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442\") " Jul 02 19:46:13.347830 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.347807 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" (UID: "a62be5bb-d2fb-4fcc-9fdd-f55338b6b442"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:46:13.347884 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.347828 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-workload-certs\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:46:13.348049 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.348027 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" (UID: "a62be5bb-d2fb-4fcc-9fdd-f55338b6b442"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:46:13.348116 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.348068 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" (UID: "a62be5bb-d2fb-4fcc-9fdd-f55338b6b442"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:46:13.348194 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.348180 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-data" (OuterVolumeSpecName: "istio-data") pod "a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" (UID: "a62be5bb-d2fb-4fcc-9fdd-f55338b6b442"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:46:13.349583 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.349555 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-kube-api-access-jwcz7" (OuterVolumeSpecName: "kube-api-access-jwcz7") pod "a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" (UID: "a62be5bb-d2fb-4fcc-9fdd-f55338b6b442"). InnerVolumeSpecName "kube-api-access-jwcz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:46:13.349701 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.349636 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" (UID: "a62be5bb-d2fb-4fcc-9fdd-f55338b6b442"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:46:13.349785 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.349767 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-token" (OuterVolumeSpecName: "istio-token") pod "a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" (UID: "a62be5bb-d2fb-4fcc-9fdd-f55338b6b442"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:46:13.349935 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.349915 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" (UID: "a62be5bb-d2fb-4fcc-9fdd-f55338b6b442"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:46:13.448328 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.448296 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-podinfo\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:46:13.448328 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.448326 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-envoy\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:46:13.448328 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.448335 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-credential-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:46:13.448556 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.448343 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istiod-ca-cert\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:46:13.448556 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.448352 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-workload-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:46:13.448556 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.448360 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-data\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:46:13.448556 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.448368 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-istio-token\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:46:13.448556 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.448376 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwcz7\" (UniqueName: \"kubernetes.io/projected/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442-kube-api-access-jwcz7\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:46:13.818476 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.818442 2556 generic.go:356] "Generic (PLEG): container finished" podID="a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" containerID="389b3e9a37bb08eaa1f3eea97fe39fbda2a82cfb5b7722764243606933c8393d" exitCode=0 Jul 02 19:46:13.818647 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.818491 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" Jul 02 19:46:13.818647 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.818531 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" event={"ID":"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442","Type":"ContainerDied","Data":"389b3e9a37bb08eaa1f3eea97fe39fbda2a82cfb5b7722764243606933c8393d"} Jul 02 19:46:13.818647 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.818611 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9" event={"ID":"a62be5bb-d2fb-4fcc-9fdd-f55338b6b442","Type":"ContainerDied","Data":"aae3ddbe1dc325d3708079409305306d6a4caa14d350ea598606906c9b8ab814"} Jul 02 19:46:13.818647 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.818637 2556 scope.go:117] "RemoveContainer" containerID="389b3e9a37bb08eaa1f3eea97fe39fbda2a82cfb5b7722764243606933c8393d" Jul 02 19:46:13.830447 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.830428 2556 scope.go:117] "RemoveContainer" containerID="33c42a52b8595f61e28c93a5a5f79bb65f52c7b77edaeebe6f4c659299fe3fc7" Jul 02 19:46:13.838349 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.838326 2556 scope.go:117] "RemoveContainer" containerID="389b3e9a37bb08eaa1f3eea97fe39fbda2a82cfb5b7722764243606933c8393d" Jul 02 19:46:13.838637 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:46:13.838617 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"389b3e9a37bb08eaa1f3eea97fe39fbda2a82cfb5b7722764243606933c8393d\": container with ID starting with 389b3e9a37bb08eaa1f3eea97fe39fbda2a82cfb5b7722764243606933c8393d not found: ID does not exist" containerID="389b3e9a37bb08eaa1f3eea97fe39fbda2a82cfb5b7722764243606933c8393d" Jul 02 19:46:13.838718 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.838650 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"389b3e9a37bb08eaa1f3eea97fe39fbda2a82cfb5b7722764243606933c8393d"} err="failed to get container status \"389b3e9a37bb08eaa1f3eea97fe39fbda2a82cfb5b7722764243606933c8393d\": rpc error: code = NotFound desc = could not find container \"389b3e9a37bb08eaa1f3eea97fe39fbda2a82cfb5b7722764243606933c8393d\": container with ID starting with 389b3e9a37bb08eaa1f3eea97fe39fbda2a82cfb5b7722764243606933c8393d not found: ID does not exist" Jul 02 19:46:13.838718 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.838675 2556 scope.go:117] "RemoveContainer" containerID="33c42a52b8595f61e28c93a5a5f79bb65f52c7b77edaeebe6f4c659299fe3fc7" Jul 02 19:46:13.838980 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:46:13.838955 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33c42a52b8595f61e28c93a5a5f79bb65f52c7b77edaeebe6f4c659299fe3fc7\": container with ID starting with 33c42a52b8595f61e28c93a5a5f79bb65f52c7b77edaeebe6f4c659299fe3fc7 not found: ID does not exist" containerID="33c42a52b8595f61e28c93a5a5f79bb65f52c7b77edaeebe6f4c659299fe3fc7" Jul 02 19:46:13.839022 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.838989 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33c42a52b8595f61e28c93a5a5f79bb65f52c7b77edaeebe6f4c659299fe3fc7"} err="failed to get container status \"33c42a52b8595f61e28c93a5a5f79bb65f52c7b77edaeebe6f4c659299fe3fc7\": rpc error: code = NotFound desc = could not find container \"33c42a52b8595f61e28c93a5a5f79bb65f52c7b77edaeebe6f4c659299fe3fc7\": container with ID starting with 33c42a52b8595f61e28c93a5a5f79bb65f52c7b77edaeebe6f4c659299fe3fc7 not found: ID does not exist" Jul 02 19:46:13.840054 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.840035 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9"] Jul 02 19:46:13.844996 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.844975 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-363f6558-openshift-default-74db8458bc-vrjq9"] Jul 02 19:46:13.978359 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:46:13.978331 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" path="/var/lib/kubelet/pods/a62be5bb-d2fb-4fcc-9fdd-f55338b6b442/volumes" Jul 02 19:47:02.735760 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.735717 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt"] Jul 02 19:47:02.736294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.735998 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b577c01d-9977-4de4-a9c5-b418e5475750" containerName="maas-api" Jul 02 19:47:02.736294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.736008 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b577c01d-9977-4de4-a9c5-b418e5475750" containerName="maas-api" Jul 02 19:47:02.736294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.736022 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" containerName="istio-proxy" Jul 02 19:47:02.736294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.736028 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" containerName="istio-proxy" Jul 02 19:47:02.736294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.736035 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" containerName="istio-proxy" Jul 02 19:47:02.736294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.736040 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" containerName="istio-proxy" Jul 02 19:47:02.736294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.736086 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" containerName="istio-proxy" Jul 02 19:47:02.736294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.736096 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b577c01d-9977-4de4-a9c5-b418e5475750" containerName="maas-api" Jul 02 19:47:02.738891 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.738872 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.742237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.742214 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-067416-openshift-default-dockercfg-ktgjf\"" Jul 02 19:47:02.763363 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.763326 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt"] Jul 02 19:47:02.850296 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.850242 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-workload-socket\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.850296 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.850297 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-token\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.850580 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.850369 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrwwg\" (UniqueName: \"kubernetes.io/projected/a39d79a1-3f00-4ac7-8b19-665bd759d87b-kube-api-access-mrwwg\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.850580 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.850421 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-podinfo\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.850580 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.850461 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-workload-certs\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.850580 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.850485 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istiod-ca-cert\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.850580 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.850518 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-data\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.850580 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.850555 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-credential-socket\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.850861 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.850609 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-envoy\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.951489 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.951446 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istiod-ca-cert\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.951690 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.951496 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-data\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.951690 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.951522 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-credential-socket\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.951690 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.951564 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-envoy\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.951690 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.951604 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-workload-socket\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.951885 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.951740 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-token\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.951885 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.951783 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrwwg\" (UniqueName: \"kubernetes.io/projected/a39d79a1-3f00-4ac7-8b19-665bd759d87b-kube-api-access-mrwwg\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.951885 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.951814 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-podinfo\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.951885 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.951856 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-workload-certs\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.952094 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.951974 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-data\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.952094 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.952022 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-credential-socket\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.952094 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.952087 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-workload-socket\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.952312 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.952244 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-workload-certs\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.952353 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.952324 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istiod-ca-cert\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.953933 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.953911 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-envoy\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.954352 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.954335 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-podinfo\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.974555 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.974527 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrwwg\" (UniqueName: \"kubernetes.io/projected/a39d79a1-3f00-4ac7-8b19-665bd759d87b-kube-api-access-mrwwg\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:02.977375 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:02.977351 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-token\") pod \"e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:03.049175 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:03.049072 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:03.199225 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:03.199188 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt"] Jul 02 19:47:03.204204 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:47:03.204178 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda39d79a1_3f00_4ac7_8b19_665bd759d87b.slice/crio-6d52012a5fe4ebdeb0c7e79b42dc7f56ca02c530c0120432db414c250652f8e6 WatchSource:0}: Error finding container 6d52012a5fe4ebdeb0c7e79b42dc7f56ca02c530c0120432db414c250652f8e6: Status 404 returned error can't find the container with id 6d52012a5fe4ebdeb0c7e79b42dc7f56ca02c530c0120432db414c250652f8e6 Jul 02 19:47:03.206317 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:03.206275 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:47:03.206401 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:03.206357 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:47:03.206445 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:03.206411 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:47:04.044823 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:04.044788 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" event={"ID":"a39d79a1-3f00-4ac7-8b19-665bd759d87b","Type":"ContainerStarted","Data":"d86df47166db609bc606c9b2049a7b5e028c06ae617656244662c43b6120852a"} Jul 02 19:47:04.045237 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:04.044840 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" event={"ID":"a39d79a1-3f00-4ac7-8b19-665bd759d87b","Type":"ContainerStarted","Data":"6d52012a5fe4ebdeb0c7e79b42dc7f56ca02c530c0120432db414c250652f8e6"} Jul 02 19:47:04.049480 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:04.049462 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:04.054191 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:04.054167 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:04.071048 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:04.070992 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" podStartSLOduration=2.070977377 podStartE2EDuration="2.070977377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:47:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:47:04.068533658 +0000 UTC m=+2136.615258665" watchObservedRunningTime="2026-07-02 19:47:04.070977377 +0000 UTC m=+2136.617702384" Jul 02 19:47:05.049229 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:05.049199 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:05.050320 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:05.050298 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:06.291630 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:06.291595 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv"] Jul 02 19:47:06.291999 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:06.291941 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a62be5bb-d2fb-4fcc-9fdd-f55338b6b442" containerName="istio-proxy" Jul 02 19:47:06.294910 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:06.294893 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" Jul 02 19:47:06.297965 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:06.297944 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-067416-serving-cert\"" Jul 02 19:47:06.298589 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:06.298570 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-wnsfc\"" Jul 02 19:47:06.322205 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:06.321854 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv"] Jul 02 19:47:06.381013 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:06.380976 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92c5574c-dadf-4cfc-97e9-be19b9703b40-maas-api-tls\") pod \"maas-api-e2e-inf-a-067416-55576cfffb-rq4rv\" (UID: \"92c5574c-dadf-4cfc-97e9-be19b9703b40\") " pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" Jul 02 19:47:06.381013 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:06.381014 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92c5574c-dadf-4cfc-97e9-be19b9703b40-openshift-service-ca\") pod \"maas-api-e2e-inf-a-067416-55576cfffb-rq4rv\" (UID: \"92c5574c-dadf-4cfc-97e9-be19b9703b40\") " pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" Jul 02 19:47:06.381312 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:06.381178 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9scb\" (UniqueName: \"kubernetes.io/projected/92c5574c-dadf-4cfc-97e9-be19b9703b40-kube-api-access-g9scb\") pod \"maas-api-e2e-inf-a-067416-55576cfffb-rq4rv\" (UID: \"92c5574c-dadf-4cfc-97e9-be19b9703b40\") " pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" Jul 02 19:47:06.482404 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:06.482355 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9scb\" (UniqueName: \"kubernetes.io/projected/92c5574c-dadf-4cfc-97e9-be19b9703b40-kube-api-access-g9scb\") pod \"maas-api-e2e-inf-a-067416-55576cfffb-rq4rv\" (UID: \"92c5574c-dadf-4cfc-97e9-be19b9703b40\") " pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" Jul 02 19:47:06.482608 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:06.482427 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92c5574c-dadf-4cfc-97e9-be19b9703b40-maas-api-tls\") pod \"maas-api-e2e-inf-a-067416-55576cfffb-rq4rv\" (UID: \"92c5574c-dadf-4cfc-97e9-be19b9703b40\") " pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" Jul 02 19:47:06.482608 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:06.482451 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92c5574c-dadf-4cfc-97e9-be19b9703b40-openshift-service-ca\") pod \"maas-api-e2e-inf-a-067416-55576cfffb-rq4rv\" (UID: \"92c5574c-dadf-4cfc-97e9-be19b9703b40\") " pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" Jul 02 19:47:06.482608 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:47:06.482584 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-067416-serving-cert: secret "maas-api-e2e-inf-a-067416-serving-cert" not found Jul 02 19:47:06.482808 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:47:06.482663 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92c5574c-dadf-4cfc-97e9-be19b9703b40-maas-api-tls podName:92c5574c-dadf-4cfc-97e9-be19b9703b40 nodeName:}" failed. No retries permitted until 2026-07-02 19:47:06.982644912 +0000 UTC m=+2139.529369907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/92c5574c-dadf-4cfc-97e9-be19b9703b40-maas-api-tls") pod "maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" (UID: "92c5574c-dadf-4cfc-97e9-be19b9703b40") : secret "maas-api-e2e-inf-a-067416-serving-cert" not found Jul 02 19:47:06.483244 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:06.483222 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92c5574c-dadf-4cfc-97e9-be19b9703b40-openshift-service-ca\") pod \"maas-api-e2e-inf-a-067416-55576cfffb-rq4rv\" (UID: \"92c5574c-dadf-4cfc-97e9-be19b9703b40\") " pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" Jul 02 19:47:06.502416 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:06.502383 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9scb\" (UniqueName: \"kubernetes.io/projected/92c5574c-dadf-4cfc-97e9-be19b9703b40-kube-api-access-g9scb\") pod \"maas-api-e2e-inf-a-067416-55576cfffb-rq4rv\" (UID: \"92c5574c-dadf-4cfc-97e9-be19b9703b40\") " pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" Jul 02 19:47:06.987485 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:06.987443 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92c5574c-dadf-4cfc-97e9-be19b9703b40-maas-api-tls\") pod \"maas-api-e2e-inf-a-067416-55576cfffb-rq4rv\" (UID: \"92c5574c-dadf-4cfc-97e9-be19b9703b40\") " pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" Jul 02 19:47:06.989920 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:06.989873 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92c5574c-dadf-4cfc-97e9-be19b9703b40-maas-api-tls\") pod \"maas-api-e2e-inf-a-067416-55576cfffb-rq4rv\" (UID: \"92c5574c-dadf-4cfc-97e9-be19b9703b40\") " pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" Jul 02 19:47:07.205319 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:07.205279 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" Jul 02 19:47:07.344698 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:07.344672 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv"] Jul 02 19:47:07.346429 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:47:07.346407 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92c5574c_dadf_4cfc_97e9_be19b9703b40.slice/crio-50f525ef20bbeb701ba9f73324c4b20b74fe2cea51262c77b65409dd2e422b9e WatchSource:0}: Error finding container 50f525ef20bbeb701ba9f73324c4b20b74fe2cea51262c77b65409dd2e422b9e: Status 404 returned error can't find the container with id 50f525ef20bbeb701ba9f73324c4b20b74fe2cea51262c77b65409dd2e422b9e Jul 02 19:47:07.348182 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:07.348140 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 19:47:08.063087 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:08.063047 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" event={"ID":"92c5574c-dadf-4cfc-97e9-be19b9703b40","Type":"ContainerStarted","Data":"50f525ef20bbeb701ba9f73324c4b20b74fe2cea51262c77b65409dd2e422b9e"} Jul 02 19:47:09.068928 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:09.068893 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" event={"ID":"92c5574c-dadf-4cfc-97e9-be19b9703b40","Type":"ContainerStarted","Data":"72f2c4441261efe5932c916378797c97689ad66f81c58715eb95eca4c6cfa5f7"} Jul 02 19:47:09.069361 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:09.069039 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" Jul 02 19:47:09.098750 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:09.098699 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" podStartSLOduration=2.353404245 podStartE2EDuration="3.098684081s" totalImagesPullingTime="745.279836ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:47:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:47:09.098666647 +0000 UTC m=+2141.645391655" watchObservedRunningTime="2026-07-02 19:47:09.098684081 +0000 UTC m=+2141.645409108" Jul 02 19:47:11.350630 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.350598 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw"] Jul 02 19:47:11.353936 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.353918 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.359824 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.359802 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-3fa1d8-openshift-default-dockercfg-czrj6\"" Jul 02 19:47:11.384460 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.384427 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw"] Jul 02 19:47:11.427071 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.427036 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60bf7063-edea-4442-bde0-ddbb3ff645d6-istiod-ca-cert\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.427281 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.427077 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-workload-socket\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.427281 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.427098 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6khb8\" (UniqueName: \"kubernetes.io/projected/60bf7063-edea-4442-bde0-ddbb3ff645d6-kube-api-access-6khb8\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.427281 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.427126 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-workload-certs\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.427281 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.427214 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-token\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.427281 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.427254 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-data\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.427281 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.427271 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-podinfo\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.427595 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.427347 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-credential-socket\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.427595 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.427440 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-envoy\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.528860 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.528821 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-credential-socket\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.528860 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.528860 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-envoy\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.529120 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.528910 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60bf7063-edea-4442-bde0-ddbb3ff645d6-istiod-ca-cert\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.529120 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.528934 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-workload-socket\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.529120 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.528959 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6khb8\" (UniqueName: \"kubernetes.io/projected/60bf7063-edea-4442-bde0-ddbb3ff645d6-kube-api-access-6khb8\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.529120 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.528988 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-workload-certs\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.529120 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.529049 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-token\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.529120 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.529089 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-data\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.529120 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.529116 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-podinfo\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.529523 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.529317 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-credential-socket\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.529523 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.529357 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-workload-socket\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.529523 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.529481 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-workload-certs\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.529638 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.529609 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-data\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.529707 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.529686 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60bf7063-edea-4442-bde0-ddbb3ff645d6-istiod-ca-cert\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.531437 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.531415 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-envoy\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.537076 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.532139 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-podinfo\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.555180 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.555129 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-token\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.555344 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.555326 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6khb8\" (UniqueName: \"kubernetes.io/projected/60bf7063-edea-4442-bde0-ddbb3ff645d6-kube-api-access-6khb8\") pod \"e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.663950 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.663857 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:11.806266 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.806237 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw"] Jul 02 19:47:11.806981 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:47:11.806953 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60bf7063_edea_4442_bde0_ddbb3ff645d6.slice/crio-7bd7969f81aae56b120d32e093b3441fe34ed07a324d2293e45e7994a1e3ce9d WatchSource:0}: Error finding container 7bd7969f81aae56b120d32e093b3441fe34ed07a324d2293e45e7994a1e3ce9d: Status 404 returned error can't find the container with id 7bd7969f81aae56b120d32e093b3441fe34ed07a324d2293e45e7994a1e3ce9d Jul 02 19:47:11.809260 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.809224 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:47:11.809330 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.809301 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:47:11.809379 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:11.809329 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:47:12.084465 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:12.084430 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" event={"ID":"60bf7063-edea-4442-bde0-ddbb3ff645d6","Type":"ContainerStarted","Data":"23ebd8f0d31085b2964f6a254357b427432a7fd46062fa36c776b5cb4883890f"} Jul 02 19:47:12.084644 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:12.084483 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" event={"ID":"60bf7063-edea-4442-bde0-ddbb3ff645d6","Type":"ContainerStarted","Data":"7bd7969f81aae56b120d32e093b3441fe34ed07a324d2293e45e7994a1e3ce9d"} Jul 02 19:47:12.135113 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:12.135047 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" podStartSLOduration=1.135026122 podStartE2EDuration="1.135026122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:47:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:47:12.132880846 +0000 UTC m=+2144.679605872" watchObservedRunningTime="2026-07-02 19:47:12.135026122 +0000 UTC m=+2144.681751129" Jul 02 19:47:12.664867 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:12.664830 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:12.670200 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:12.670175 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:13.089568 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:13.089542 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:13.090546 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:13.090521 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:13.409539 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:13.409455 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:47:15.079112 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:15.079080 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" Jul 02 19:47:21.092812 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:21.092364 2556 patch_prober.go:28] interesting pod/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 02 19:47:21.092812 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:21.092482 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" podUID="a39d79a1-3f00-4ac7-8b19-665bd759d87b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 02 19:47:22.140271 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:22.140237 2556 generic.go:356] "Generic (PLEG): container finished" podID="a39d79a1-3f00-4ac7-8b19-665bd759d87b" containerID="d86df47166db609bc606c9b2049a7b5e028c06ae617656244662c43b6120852a" exitCode=0 Jul 02 19:47:22.140271 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:22.140275 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" event={"ID":"a39d79a1-3f00-4ac7-8b19-665bd759d87b","Type":"ContainerDied","Data":"d86df47166db609bc606c9b2049a7b5e028c06ae617656244662c43b6120852a"} Jul 02 19:47:22.140697 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:22.140646 2556 scope.go:117] "RemoveContainer" containerID="d86df47166db609bc606c9b2049a7b5e028c06ae617656244662c43b6120852a" Jul 02 19:47:22.141743 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:22.141715 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:47:22.141807 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:22.141790 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:47:22.141843 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:22.141827 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:47:23.147745 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:23.147710 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" event={"ID":"a39d79a1-3f00-4ac7-8b19-665bd759d87b","Type":"ContainerStarted","Data":"088d145712f1cbecd82d793b4dcdcc6df168d487ea0363783e5f574795c8a960"} Jul 02 19:47:24.049804 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:24.049765 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:24.050043 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:24.050018 2556 patch_prober.go:28] interesting pod/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 02 19:47:24.050128 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:24.050074 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" podUID="a39d79a1-3f00-4ac7-8b19-665bd759d87b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 02 19:47:24.299071 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:24.299027 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:47:25.050502 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:25.050459 2556 patch_prober.go:28] interesting pod/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 02 19:47:25.050692 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:25.050518 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" podUID="a39d79a1-3f00-4ac7-8b19-665bd759d87b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 02 19:47:26.050079 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:26.050038 2556 patch_prober.go:28] interesting pod/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 02 19:47:26.050561 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:26.050098 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" podUID="a39d79a1-3f00-4ac7-8b19-665bd759d87b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 02 19:47:27.057662 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:27.057622 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:27.058197 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:27.058174 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:27.058988 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:27.058963 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:47:28.111407 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:28.111379 2556 patch_prober.go:28] interesting pod/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": read tcp 10.132.0.2:34344->10.132.0.40:15021: read: connection reset by peer" start-of-body= Jul 02 19:47:28.111815 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:28.111445 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" podUID="60bf7063-edea-4442-bde0-ddbb3ff645d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": read tcp 10.132.0.2:34344->10.132.0.40:15021: read: connection reset by peer" Jul 02 19:47:28.171594 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:28.171568 2556 generic.go:356] "Generic (PLEG): container finished" podID="60bf7063-edea-4442-bde0-ddbb3ff645d6" containerID="23ebd8f0d31085b2964f6a254357b427432a7fd46062fa36c776b5cb4883890f" exitCode=0 Jul 02 19:47:28.171690 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:28.171640 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" event={"ID":"60bf7063-edea-4442-bde0-ddbb3ff645d6","Type":"ContainerDied","Data":"23ebd8f0d31085b2964f6a254357b427432a7fd46062fa36c776b5cb4883890f"} Jul 02 19:47:28.172130 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:28.172106 2556 scope.go:117] "RemoveContainer" containerID="23ebd8f0d31085b2964f6a254357b427432a7fd46062fa36c776b5cb4883890f" Jul 02 19:47:28.173020 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:28.172993 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:47:28.173090 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:28.173065 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:47:28.173127 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:28.173099 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:47:29.178350 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:29.178318 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" event={"ID":"60bf7063-edea-4442-bde0-ddbb3ff645d6","Type":"ContainerStarted","Data":"d3b5eabbe8d54a12377903dc3f6eb997649c1d221856021e1aca66578f86d3ca"} Jul 02 19:47:29.665059 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:29.664967 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:29.665236 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:29.665185 2556 patch_prober.go:28] interesting pod/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 02 19:47:29.665283 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:29.665238 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" podUID="60bf7063-edea-4442-bde0-ddbb3ff645d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 02 19:47:30.665248 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:30.665208 2556 patch_prober.go:28] interesting pod/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 02 19:47:30.665641 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:30.665267 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" podUID="60bf7063-edea-4442-bde0-ddbb3ff645d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 02 19:47:31.664587 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:31.664550 2556 patch_prober.go:28] interesting pod/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 02 19:47:31.664776 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:31.664604 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" podUID="60bf7063-edea-4442-bde0-ddbb3ff645d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 02 19:47:32.664721 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:32.664685 2556 patch_prober.go:28] interesting pod/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 02 19:47:32.665107 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:32.664749 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" podUID="60bf7063-edea-4442-bde0-ddbb3ff645d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 02 19:47:33.667690 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:33.667656 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:33.668205 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:33.668062 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:33.668645 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:33.668628 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:47:44.417886 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:44.417848 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:47:55.416982 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:47:55.416943 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:48:21.133245 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:48:21.133208 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:48:35.505270 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:48:35.505234 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:49:05.099416 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.099320 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv"] Jul 02 19:49:05.105002 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.104954 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" podUID="92c5574c-dadf-4cfc-97e9-be19b9703b40" containerName="maas-api" containerID="cri-o://72f2c4441261efe5932c916378797c97689ad66f81c58715eb95eca4c6cfa5f7" gracePeriod=30 Jul 02 19:49:05.352186 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.352106 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" Jul 02 19:49:05.497447 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.497409 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92c5574c-dadf-4cfc-97e9-be19b9703b40-maas-api-tls\") pod \"92c5574c-dadf-4cfc-97e9-be19b9703b40\" (UID: \"92c5574c-dadf-4cfc-97e9-be19b9703b40\") " Jul 02 19:49:05.497447 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.497462 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9scb\" (UniqueName: \"kubernetes.io/projected/92c5574c-dadf-4cfc-97e9-be19b9703b40-kube-api-access-g9scb\") pod \"92c5574c-dadf-4cfc-97e9-be19b9703b40\" (UID: \"92c5574c-dadf-4cfc-97e9-be19b9703b40\") " Jul 02 19:49:05.497711 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.497490 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92c5574c-dadf-4cfc-97e9-be19b9703b40-openshift-service-ca\") pod \"92c5574c-dadf-4cfc-97e9-be19b9703b40\" (UID: \"92c5574c-dadf-4cfc-97e9-be19b9703b40\") " Jul 02 19:49:05.497936 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.497900 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92c5574c-dadf-4cfc-97e9-be19b9703b40-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "92c5574c-dadf-4cfc-97e9-be19b9703b40" (UID: "92c5574c-dadf-4cfc-97e9-be19b9703b40"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:49:05.499652 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.499628 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92c5574c-dadf-4cfc-97e9-be19b9703b40-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "92c5574c-dadf-4cfc-97e9-be19b9703b40" (UID: "92c5574c-dadf-4cfc-97e9-be19b9703b40"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:49:05.499750 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.499660 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92c5574c-dadf-4cfc-97e9-be19b9703b40-kube-api-access-g9scb" (OuterVolumeSpecName: "kube-api-access-g9scb") pod "92c5574c-dadf-4cfc-97e9-be19b9703b40" (UID: "92c5574c-dadf-4cfc-97e9-be19b9703b40"). InnerVolumeSpecName "kube-api-access-g9scb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:49:05.598550 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.598496 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92c5574c-dadf-4cfc-97e9-be19b9703b40-maas-api-tls\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:05.598550 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.598545 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9scb\" (UniqueName: \"kubernetes.io/projected/92c5574c-dadf-4cfc-97e9-be19b9703b40-kube-api-access-g9scb\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:05.598550 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.598558 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92c5574c-dadf-4cfc-97e9-be19b9703b40-openshift-service-ca\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:05.631499 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.631415 2556 generic.go:356] "Generic (PLEG): container finished" podID="92c5574c-dadf-4cfc-97e9-be19b9703b40" containerID="72f2c4441261efe5932c916378797c97689ad66f81c58715eb95eca4c6cfa5f7" exitCode=0 Jul 02 19:49:05.631499 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.631462 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" Jul 02 19:49:05.631660 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.631502 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" event={"ID":"92c5574c-dadf-4cfc-97e9-be19b9703b40","Type":"ContainerDied","Data":"72f2c4441261efe5932c916378797c97689ad66f81c58715eb95eca4c6cfa5f7"} Jul 02 19:49:05.631660 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.631559 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv" event={"ID":"92c5574c-dadf-4cfc-97e9-be19b9703b40","Type":"ContainerDied","Data":"50f525ef20bbeb701ba9f73324c4b20b74fe2cea51262c77b65409dd2e422b9e"} Jul 02 19:49:05.631660 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.631581 2556 scope.go:117] "RemoveContainer" containerID="72f2c4441261efe5932c916378797c97689ad66f81c58715eb95eca4c6cfa5f7" Jul 02 19:49:05.643061 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.643040 2556 scope.go:117] "RemoveContainer" containerID="72f2c4441261efe5932c916378797c97689ad66f81c58715eb95eca4c6cfa5f7" Jul 02 19:49:05.643361 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:49:05.643336 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72f2c4441261efe5932c916378797c97689ad66f81c58715eb95eca4c6cfa5f7\": container with ID starting with 72f2c4441261efe5932c916378797c97689ad66f81c58715eb95eca4c6cfa5f7 not found: ID does not exist" containerID="72f2c4441261efe5932c916378797c97689ad66f81c58715eb95eca4c6cfa5f7" Jul 02 19:49:05.643472 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.643372 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72f2c4441261efe5932c916378797c97689ad66f81c58715eb95eca4c6cfa5f7"} err="failed to get container status \"72f2c4441261efe5932c916378797c97689ad66f81c58715eb95eca4c6cfa5f7\": rpc error: code = NotFound desc = could not find container \"72f2c4441261efe5932c916378797c97689ad66f81c58715eb95eca4c6cfa5f7\": container with ID starting with 72f2c4441261efe5932c916378797c97689ad66f81c58715eb95eca4c6cfa5f7 not found: ID does not exist" Jul 02 19:49:05.653990 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.653965 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv"] Jul 02 19:49:05.660718 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.660697 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-067416-55576cfffb-rq4rv"] Jul 02 19:49:05.978435 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:05.978406 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92c5574c-dadf-4cfc-97e9-be19b9703b40" path="/var/lib/kubelet/pods/92c5574c-dadf-4cfc-97e9-be19b9703b40/volumes" Jul 02 19:49:08.002724 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:08.002692 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:49:46.816135 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:46.816103 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt"] Jul 02 19:49:46.816640 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:46.816333 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" podUID="a39d79a1-3f00-4ac7-8b19-665bd759d87b" containerName="istio-proxy" containerID="cri-o://088d145712f1cbecd82d793b4dcdcc6df168d487ea0363783e5f574795c8a960" gracePeriod=30 Jul 02 19:49:52.068811 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.068782 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:49:52.182586 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.182555 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istiod-ca-cert\") pod \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " Jul 02 19:49:52.182853 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.182600 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-workload-certs\") pod \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " Jul 02 19:49:52.182853 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.182627 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-token\") pod \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " Jul 02 19:49:52.182853 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.182645 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-envoy\") pod \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " Jul 02 19:49:52.182853 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.182671 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrwwg\" (UniqueName: \"kubernetes.io/projected/a39d79a1-3f00-4ac7-8b19-665bd759d87b-kube-api-access-mrwwg\") pod \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " Jul 02 19:49:52.182853 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.182712 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-data\") pod \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " Jul 02 19:49:52.182853 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.182754 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-workload-socket\") pod \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " Jul 02 19:49:52.182853 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.182779 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-podinfo\") pod \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " Jul 02 19:49:52.182853 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.182817 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-credential-socket\") pod \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\" (UID: \"a39d79a1-3f00-4ac7-8b19-665bd759d87b\") " Jul 02 19:49:52.183272 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.182939 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a39d79a1-3f00-4ac7-8b19-665bd759d87b" (UID: "a39d79a1-3f00-4ac7-8b19-665bd759d87b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:49:52.183272 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.183080 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-workload-certs\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:52.183272 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.183006 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a39d79a1-3f00-4ac7-8b19-665bd759d87b" (UID: "a39d79a1-3f00-4ac7-8b19-665bd759d87b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:49:52.183272 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.183084 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a39d79a1-3f00-4ac7-8b19-665bd759d87b" (UID: "a39d79a1-3f00-4ac7-8b19-665bd759d87b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:49:52.183496 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.183463 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a39d79a1-3f00-4ac7-8b19-665bd759d87b" (UID: "a39d79a1-3f00-4ac7-8b19-665bd759d87b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:49:52.183811 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.183791 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-data" (OuterVolumeSpecName: "istio-data") pod "a39d79a1-3f00-4ac7-8b19-665bd759d87b" (UID: "a39d79a1-3f00-4ac7-8b19-665bd759d87b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:49:52.185081 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.185054 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a39d79a1-3f00-4ac7-8b19-665bd759d87b-kube-api-access-mrwwg" (OuterVolumeSpecName: "kube-api-access-mrwwg") pod "a39d79a1-3f00-4ac7-8b19-665bd759d87b" (UID: "a39d79a1-3f00-4ac7-8b19-665bd759d87b"). InnerVolumeSpecName "kube-api-access-mrwwg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:49:52.185081 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.185074 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-token" (OuterVolumeSpecName: "istio-token") pod "a39d79a1-3f00-4ac7-8b19-665bd759d87b" (UID: "a39d79a1-3f00-4ac7-8b19-665bd759d87b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:49:52.185378 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.185359 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a39d79a1-3f00-4ac7-8b19-665bd759d87b" (UID: "a39d79a1-3f00-4ac7-8b19-665bd759d87b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:49:52.185460 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.185362 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a39d79a1-3f00-4ac7-8b19-665bd759d87b" (UID: "a39d79a1-3f00-4ac7-8b19-665bd759d87b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:49:52.283751 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.283715 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istiod-ca-cert\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:52.283751 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.283744 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-token\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:52.283751 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.283753 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-envoy\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:52.283991 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.283761 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrwwg\" (UniqueName: \"kubernetes.io/projected/a39d79a1-3f00-4ac7-8b19-665bd759d87b-kube-api-access-mrwwg\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:52.283991 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.283773 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-data\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:52.283991 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.283781 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-workload-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:52.283991 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.283790 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a39d79a1-3f00-4ac7-8b19-665bd759d87b-istio-podinfo\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:52.283991 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.283798 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a39d79a1-3f00-4ac7-8b19-665bd759d87b-credential-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:52.858073 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.858038 2556 generic.go:356] "Generic (PLEG): container finished" podID="a39d79a1-3f00-4ac7-8b19-665bd759d87b" containerID="088d145712f1cbecd82d793b4dcdcc6df168d487ea0363783e5f574795c8a960" exitCode=0 Jul 02 19:49:52.858260 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.858088 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" Jul 02 19:49:52.858260 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.858109 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" event={"ID":"a39d79a1-3f00-4ac7-8b19-665bd759d87b","Type":"ContainerDied","Data":"088d145712f1cbecd82d793b4dcdcc6df168d487ea0363783e5f574795c8a960"} Jul 02 19:49:52.858260 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.858164 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt" event={"ID":"a39d79a1-3f00-4ac7-8b19-665bd759d87b","Type":"ContainerDied","Data":"6d52012a5fe4ebdeb0c7e79b42dc7f56ca02c530c0120432db414c250652f8e6"} Jul 02 19:49:52.858260 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.858186 2556 scope.go:117] "RemoveContainer" containerID="088d145712f1cbecd82d793b4dcdcc6df168d487ea0363783e5f574795c8a960" Jul 02 19:49:52.867637 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.867620 2556 scope.go:117] "RemoveContainer" containerID="d86df47166db609bc606c9b2049a7b5e028c06ae617656244662c43b6120852a" Jul 02 19:49:52.875341 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.875327 2556 scope.go:117] "RemoveContainer" containerID="088d145712f1cbecd82d793b4dcdcc6df168d487ea0363783e5f574795c8a960" Jul 02 19:49:52.875574 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:49:52.875555 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"088d145712f1cbecd82d793b4dcdcc6df168d487ea0363783e5f574795c8a960\": container with ID starting with 088d145712f1cbecd82d793b4dcdcc6df168d487ea0363783e5f574795c8a960 not found: ID does not exist" containerID="088d145712f1cbecd82d793b4dcdcc6df168d487ea0363783e5f574795c8a960" Jul 02 19:49:52.875621 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.875582 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"088d145712f1cbecd82d793b4dcdcc6df168d487ea0363783e5f574795c8a960"} err="failed to get container status \"088d145712f1cbecd82d793b4dcdcc6df168d487ea0363783e5f574795c8a960\": rpc error: code = NotFound desc = could not find container \"088d145712f1cbecd82d793b4dcdcc6df168d487ea0363783e5f574795c8a960\": container with ID starting with 088d145712f1cbecd82d793b4dcdcc6df168d487ea0363783e5f574795c8a960 not found: ID does not exist" Jul 02 19:49:52.875621 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.875600 2556 scope.go:117] "RemoveContainer" containerID="d86df47166db609bc606c9b2049a7b5e028c06ae617656244662c43b6120852a" Jul 02 19:49:52.875793 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:49:52.875774 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d86df47166db609bc606c9b2049a7b5e028c06ae617656244662c43b6120852a\": container with ID starting with d86df47166db609bc606c9b2049a7b5e028c06ae617656244662c43b6120852a not found: ID does not exist" containerID="d86df47166db609bc606c9b2049a7b5e028c06ae617656244662c43b6120852a" Jul 02 19:49:52.875837 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.875797 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d86df47166db609bc606c9b2049a7b5e028c06ae617656244662c43b6120852a"} err="failed to get container status \"d86df47166db609bc606c9b2049a7b5e028c06ae617656244662c43b6120852a\": rpc error: code = NotFound desc = could not find container \"d86df47166db609bc606c9b2049a7b5e028c06ae617656244662c43b6120852a\": container with ID starting with d86df47166db609bc606c9b2049a7b5e028c06ae617656244662c43b6120852a not found: ID does not exist" Jul 02 19:49:52.885349 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.885318 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt"] Jul 02 19:49:52.886967 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:52.886946 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-067416-openshift-default-6ccc987785-wxhlt"] Jul 02 19:49:53.202450 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:53.202408 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:49:53.838467 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:53.838423 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw"] Jul 02 19:49:53.838765 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:53.838727 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" podUID="60bf7063-edea-4442-bde0-ddbb3ff645d6" containerName="istio-proxy" containerID="cri-o://d3b5eabbe8d54a12377903dc3f6eb997649c1d221856021e1aca66578f86d3ca" gracePeriod=30 Jul 02 19:49:53.977663 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:53.977631 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a39d79a1-3f00-4ac7-8b19-665bd759d87b" path="/var/lib/kubelet/pods/a39d79a1-3f00-4ac7-8b19-665bd759d87b/volumes" Jul 02 19:49:54.735724 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.735691 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx"] Jul 02 19:49:54.736084 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.735971 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a39d79a1-3f00-4ac7-8b19-665bd759d87b" containerName="istio-proxy" Jul 02 19:49:54.736084 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.735982 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a39d79a1-3f00-4ac7-8b19-665bd759d87b" containerName="istio-proxy" Jul 02 19:49:54.736084 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.735996 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a39d79a1-3f00-4ac7-8b19-665bd759d87b" containerName="istio-proxy" Jul 02 19:49:54.736084 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.736002 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a39d79a1-3f00-4ac7-8b19-665bd759d87b" containerName="istio-proxy" Jul 02 19:49:54.736084 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.736015 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92c5574c-dadf-4cfc-97e9-be19b9703b40" containerName="maas-api" Jul 02 19:49:54.736084 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.736021 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="92c5574c-dadf-4cfc-97e9-be19b9703b40" containerName="maas-api" Jul 02 19:49:54.736084 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.736069 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="92c5574c-dadf-4cfc-97e9-be19b9703b40" containerName="maas-api" Jul 02 19:49:54.736084 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.736077 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a39d79a1-3f00-4ac7-8b19-665bd759d87b" containerName="istio-proxy" Jul 02 19:49:54.736084 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.736084 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a39d79a1-3f00-4ac7-8b19-665bd759d87b" containerName="istio-proxy" Jul 02 19:49:54.738652 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.738637 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.741602 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.741579 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-455cd8-openshift-default-dockercfg-h5pq2\"" Jul 02 19:49:54.750276 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.750252 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx"] Jul 02 19:49:54.804763 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.804731 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-podinfo\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.804933 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.804770 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f935e321-f1e8-4f4e-971f-5693823afdbf-istiod-ca-cert\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.804933 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.804792 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-envoy\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.804933 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.804844 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-workload-socket\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.804933 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.804923 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-workload-certs\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.805082 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.804999 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-credential-socket\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.805082 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.805028 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-data\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.805082 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.805048 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xztt7\" (UniqueName: \"kubernetes.io/projected/f935e321-f1e8-4f4e-971f-5693823afdbf-kube-api-access-xztt7\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.805082 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.805070 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-token\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.905483 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.905446 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-credential-socket\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.905483 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.905484 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-data\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.905703 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.905502 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xztt7\" (UniqueName: \"kubernetes.io/projected/f935e321-f1e8-4f4e-971f-5693823afdbf-kube-api-access-xztt7\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.905703 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.905623 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-token\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.905782 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.905723 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-podinfo\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.905782 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.905768 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f935e321-f1e8-4f4e-971f-5693823afdbf-istiod-ca-cert\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.905867 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.905801 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-envoy\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.905867 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.905837 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-workload-socket\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.905867 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.905855 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-credential-socket\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.905867 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.905864 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-workload-certs\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.906070 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.905909 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-data\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.906129 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.906087 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-workload-socket\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.906274 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.906250 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-workload-certs\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.906463 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.906446 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f935e321-f1e8-4f4e-971f-5693823afdbf-istiod-ca-cert\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.908090 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.908072 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-envoy\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.908336 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.908318 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-podinfo\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.914041 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.914018 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-token\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:54.914225 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:54.914197 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xztt7\" (UniqueName: \"kubernetes.io/projected/f935e321-f1e8-4f4e-971f-5693823afdbf-kube-api-access-xztt7\") pod \"e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:55.049619 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:55.049522 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:55.190312 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:55.190287 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx"] Jul 02 19:49:55.192388 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:49:55.192359 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf935e321_f1e8_4f4e_971f_5693823afdbf.slice/crio-5f3ad7e46c6bcad864ae27ad11d7ae87961fb65fda05be6790ed9b634d1e6d0f WatchSource:0}: Error finding container 5f3ad7e46c6bcad864ae27ad11d7ae87961fb65fda05be6790ed9b634d1e6d0f: Status 404 returned error can't find the container with id 5f3ad7e46c6bcad864ae27ad11d7ae87961fb65fda05be6790ed9b634d1e6d0f Jul 02 19:49:55.194587 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:55.194550 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:49:55.194653 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:55.194635 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:49:55.194701 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:55.194678 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:49:55.874205 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:55.874168 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" event={"ID":"f935e321-f1e8-4f4e-971f-5693823afdbf","Type":"ContainerStarted","Data":"7afdad8037b0ade7e37f245b2cd5560c256cecb1eaa5630e1e57d28627a33f49"} Jul 02 19:49:55.874615 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:55.874229 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" event={"ID":"f935e321-f1e8-4f4e-971f-5693823afdbf","Type":"ContainerStarted","Data":"5f3ad7e46c6bcad864ae27ad11d7ae87961fb65fda05be6790ed9b634d1e6d0f"} Jul 02 19:49:55.896325 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:55.896277 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" podStartSLOduration=1.896260895 podStartE2EDuration="1.896260895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:49:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:49:55.895127279 +0000 UTC m=+2308.441852288" watchObservedRunningTime="2026-07-02 19:49:55.896260895 +0000 UTC m=+2308.442985901" Jul 02 19:49:56.049951 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:56.049918 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:56.054766 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:56.054739 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:56.879233 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:56.879200 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:56.880168 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:56.880131 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:49:59.085196 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.085172 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:49:59.145440 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.145349 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60bf7063-edea-4442-bde0-ddbb3ff645d6-istiod-ca-cert\") pod \"60bf7063-edea-4442-bde0-ddbb3ff645d6\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " Jul 02 19:49:59.145440 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.145400 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-workload-certs\") pod \"60bf7063-edea-4442-bde0-ddbb3ff645d6\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " Jul 02 19:49:59.145440 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.145426 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-credential-socket\") pod \"60bf7063-edea-4442-bde0-ddbb3ff645d6\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " Jul 02 19:49:59.145698 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.145450 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-podinfo\") pod \"60bf7063-edea-4442-bde0-ddbb3ff645d6\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " Jul 02 19:49:59.145698 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.145612 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-envoy\") pod \"60bf7063-edea-4442-bde0-ddbb3ff645d6\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " Jul 02 19:49:59.145698 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.145690 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-data\") pod \"60bf7063-edea-4442-bde0-ddbb3ff645d6\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " Jul 02 19:49:59.145855 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.145722 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6khb8\" (UniqueName: \"kubernetes.io/projected/60bf7063-edea-4442-bde0-ddbb3ff645d6-kube-api-access-6khb8\") pod \"60bf7063-edea-4442-bde0-ddbb3ff645d6\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " Jul 02 19:49:59.145855 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.145737 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "60bf7063-edea-4442-bde0-ddbb3ff645d6" (UID: "60bf7063-edea-4442-bde0-ddbb3ff645d6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:49:59.145855 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.145752 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "60bf7063-edea-4442-bde0-ddbb3ff645d6" (UID: "60bf7063-edea-4442-bde0-ddbb3ff645d6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:49:59.145855 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.145747 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-token\") pod \"60bf7063-edea-4442-bde0-ddbb3ff645d6\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " Jul 02 19:49:59.145855 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.145820 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60bf7063-edea-4442-bde0-ddbb3ff645d6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "60bf7063-edea-4442-bde0-ddbb3ff645d6" (UID: "60bf7063-edea-4442-bde0-ddbb3ff645d6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:49:59.145855 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.145823 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-workload-socket\") pod \"60bf7063-edea-4442-bde0-ddbb3ff645d6\" (UID: \"60bf7063-edea-4442-bde0-ddbb3ff645d6\") " Jul 02 19:49:59.146199 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.146034 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "60bf7063-edea-4442-bde0-ddbb3ff645d6" (UID: "60bf7063-edea-4442-bde0-ddbb3ff645d6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:49:59.146262 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.146209 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-workload-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:59.146262 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.146223 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60bf7063-edea-4442-bde0-ddbb3ff645d6-istiod-ca-cert\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:59.146262 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.146232 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-workload-certs\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:59.146262 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.146241 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-credential-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:59.146538 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.146512 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-data" (OuterVolumeSpecName: "istio-data") pod "60bf7063-edea-4442-bde0-ddbb3ff645d6" (UID: "60bf7063-edea-4442-bde0-ddbb3ff645d6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:49:59.147763 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.147735 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "60bf7063-edea-4442-bde0-ddbb3ff645d6" (UID: "60bf7063-edea-4442-bde0-ddbb3ff645d6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:49:59.147763 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.147745 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "60bf7063-edea-4442-bde0-ddbb3ff645d6" (UID: "60bf7063-edea-4442-bde0-ddbb3ff645d6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:49:59.148201 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.148177 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-token" (OuterVolumeSpecName: "istio-token") pod "60bf7063-edea-4442-bde0-ddbb3ff645d6" (UID: "60bf7063-edea-4442-bde0-ddbb3ff645d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:49:59.148312 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.148210 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60bf7063-edea-4442-bde0-ddbb3ff645d6-kube-api-access-6khb8" (OuterVolumeSpecName: "kube-api-access-6khb8") pod "60bf7063-edea-4442-bde0-ddbb3ff645d6" (UID: "60bf7063-edea-4442-bde0-ddbb3ff645d6"). InnerVolumeSpecName "kube-api-access-6khb8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:49:59.247495 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.247457 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-data\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:59.247495 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.247489 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6khb8\" (UniqueName: \"kubernetes.io/projected/60bf7063-edea-4442-bde0-ddbb3ff645d6-kube-api-access-6khb8\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:59.247495 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.247501 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-token\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:59.247726 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.247509 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-podinfo\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:59.247726 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.247517 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60bf7063-edea-4442-bde0-ddbb3ff645d6-istio-envoy\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:49:59.895412 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.895380 2556 generic.go:356] "Generic (PLEG): container finished" podID="60bf7063-edea-4442-bde0-ddbb3ff645d6" containerID="d3b5eabbe8d54a12377903dc3f6eb997649c1d221856021e1aca66578f86d3ca" exitCode=0 Jul 02 19:49:59.895577 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.895429 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" Jul 02 19:49:59.895577 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.895472 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" event={"ID":"60bf7063-edea-4442-bde0-ddbb3ff645d6","Type":"ContainerDied","Data":"d3b5eabbe8d54a12377903dc3f6eb997649c1d221856021e1aca66578f86d3ca"} Jul 02 19:49:59.895577 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.895527 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw" event={"ID":"60bf7063-edea-4442-bde0-ddbb3ff645d6","Type":"ContainerDied","Data":"7bd7969f81aae56b120d32e093b3441fe34ed07a324d2293e45e7994a1e3ce9d"} Jul 02 19:49:59.895577 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.895548 2556 scope.go:117] "RemoveContainer" containerID="d3b5eabbe8d54a12377903dc3f6eb997649c1d221856021e1aca66578f86d3ca" Jul 02 19:49:59.905256 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.905235 2556 scope.go:117] "RemoveContainer" containerID="23ebd8f0d31085b2964f6a254357b427432a7fd46062fa36c776b5cb4883890f" Jul 02 19:49:59.912757 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.912741 2556 scope.go:117] "RemoveContainer" containerID="d3b5eabbe8d54a12377903dc3f6eb997649c1d221856021e1aca66578f86d3ca" Jul 02 19:49:59.913015 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:49:59.912997 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3b5eabbe8d54a12377903dc3f6eb997649c1d221856021e1aca66578f86d3ca\": container with ID starting with d3b5eabbe8d54a12377903dc3f6eb997649c1d221856021e1aca66578f86d3ca not found: ID does not exist" containerID="d3b5eabbe8d54a12377903dc3f6eb997649c1d221856021e1aca66578f86d3ca" Jul 02 19:49:59.913062 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.913023 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3b5eabbe8d54a12377903dc3f6eb997649c1d221856021e1aca66578f86d3ca"} err="failed to get container status \"d3b5eabbe8d54a12377903dc3f6eb997649c1d221856021e1aca66578f86d3ca\": rpc error: code = NotFound desc = could not find container \"d3b5eabbe8d54a12377903dc3f6eb997649c1d221856021e1aca66578f86d3ca\": container with ID starting with d3b5eabbe8d54a12377903dc3f6eb997649c1d221856021e1aca66578f86d3ca not found: ID does not exist" Jul 02 19:49:59.913062 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.913042 2556 scope.go:117] "RemoveContainer" containerID="23ebd8f0d31085b2964f6a254357b427432a7fd46062fa36c776b5cb4883890f" Jul 02 19:49:59.913285 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:49:59.913268 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23ebd8f0d31085b2964f6a254357b427432a7fd46062fa36c776b5cb4883890f\": container with ID starting with 23ebd8f0d31085b2964f6a254357b427432a7fd46062fa36c776b5cb4883890f not found: ID does not exist" containerID="23ebd8f0d31085b2964f6a254357b427432a7fd46062fa36c776b5cb4883890f" Jul 02 19:49:59.913331 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.913291 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23ebd8f0d31085b2964f6a254357b427432a7fd46062fa36c776b5cb4883890f"} err="failed to get container status \"23ebd8f0d31085b2964f6a254357b427432a7fd46062fa36c776b5cb4883890f\": rpc error: code = NotFound desc = could not find container \"23ebd8f0d31085b2964f6a254357b427432a7fd46062fa36c776b5cb4883890f\": container with ID starting with 23ebd8f0d31085b2964f6a254357b427432a7fd46062fa36c776b5cb4883890f not found: ID does not exist" Jul 02 19:49:59.918883 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.918852 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw"] Jul 02 19:49:59.926958 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.926938 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-3fa1d8-openshift-default-5d8fb8cbb5-qx2tw"] Jul 02 19:49:59.979374 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:49:59.979347 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60bf7063-edea-4442-bde0-ddbb3ff645d6" path="/var/lib/kubelet/pods/60bf7063-edea-4442-bde0-ddbb3ff645d6/volumes" Jul 02 19:50:03.119397 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.119363 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5"] Jul 02 19:50:03.119784 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.119728 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60bf7063-edea-4442-bde0-ddbb3ff645d6" containerName="istio-proxy" Jul 02 19:50:03.119784 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.119745 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="60bf7063-edea-4442-bde0-ddbb3ff645d6" containerName="istio-proxy" Jul 02 19:50:03.119784 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.119769 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60bf7063-edea-4442-bde0-ddbb3ff645d6" containerName="istio-proxy" Jul 02 19:50:03.119784 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.119779 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="60bf7063-edea-4442-bde0-ddbb3ff645d6" containerName="istio-proxy" Jul 02 19:50:03.119964 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.119853 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="60bf7063-edea-4442-bde0-ddbb3ff645d6" containerName="istio-proxy" Jul 02 19:50:03.123134 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.123084 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.125900 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.125875 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-1d5ad8-openshift-default-dockercfg-kppsf\"" Jul 02 19:50:03.136450 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.136426 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5"] Jul 02 19:50:03.282700 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.282658 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-token\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.282700 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.282698 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-workload-certs\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.282917 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.282730 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-workload-socket\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.282917 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.282800 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2q55\" (UniqueName: \"kubernetes.io/projected/290ec0dc-cfc7-46c8-a865-17f68f47701e-kube-api-access-n2q55\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.282917 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.282835 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-podinfo\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.282917 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.282887 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-credential-socket\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.282917 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.282907 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/290ec0dc-cfc7-46c8-a865-17f68f47701e-istiod-ca-cert\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.283075 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.282927 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-envoy\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.283075 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.282981 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-data\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.383932 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.383821 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-token\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.383932 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.383864 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-workload-certs\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.383932 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.383895 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-workload-socket\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.383932 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.383914 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2q55\" (UniqueName: \"kubernetes.io/projected/290ec0dc-cfc7-46c8-a865-17f68f47701e-kube-api-access-n2q55\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.384328 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.383968 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-podinfo\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.384328 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.384035 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-credential-socket\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.384328 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.384056 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/290ec0dc-cfc7-46c8-a865-17f68f47701e-istiod-ca-cert\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.384328 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.384087 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-envoy\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.384328 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.384295 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-data\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.384556 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.384375 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-workload-certs\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.384556 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.384432 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-credential-socket\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.384556 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.384431 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-workload-socket\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.384661 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.384618 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-data\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.384913 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.384894 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/290ec0dc-cfc7-46c8-a865-17f68f47701e-istiod-ca-cert\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.386435 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.386409 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-envoy\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.386534 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.386517 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-podinfo\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.392194 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.392171 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2q55\" (UniqueName: \"kubernetes.io/projected/290ec0dc-cfc7-46c8-a865-17f68f47701e-kube-api-access-n2q55\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.392278 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.392172 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-token\") pod \"e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.433357 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.433308 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:03.566605 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.566579 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5"] Jul 02 19:50:03.568727 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:50:03.568699 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod290ec0dc_cfc7_46c8_a865_17f68f47701e.slice/crio-10ba649acf925f6851e59a754952cb6e86a6bf3a93bc347647724ef962d709e8 WatchSource:0}: Error finding container 10ba649acf925f6851e59a754952cb6e86a6bf3a93bc347647724ef962d709e8: Status 404 returned error can't find the container with id 10ba649acf925f6851e59a754952cb6e86a6bf3a93bc347647724ef962d709e8 Jul 02 19:50:03.570867 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.570817 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:03.570967 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.570898 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:03.570967 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.570926 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:03.915652 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.915616 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" event={"ID":"290ec0dc-cfc7-46c8-a865-17f68f47701e","Type":"ContainerStarted","Data":"9bab85cfc451558f6daef564d2ff136f23ae01525ce54a42ed64c6ebfe35aa73"} Jul 02 19:50:03.915877 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.915679 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" event={"ID":"290ec0dc-cfc7-46c8-a865-17f68f47701e","Type":"ContainerStarted","Data":"10ba649acf925f6851e59a754952cb6e86a6bf3a93bc347647724ef962d709e8"} Jul 02 19:50:03.935293 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:03.935241 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" podStartSLOduration=0.935219773 podStartE2EDuration="935.219773ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:50:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:50:03.934283679 +0000 UTC m=+2316.481008708" watchObservedRunningTime="2026-07-02 19:50:03.935219773 +0000 UTC m=+2316.481944780" Jul 02 19:50:04.433924 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:04.433876 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:04.439087 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:04.439060 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:04.920584 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:04.920545 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:04.921630 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:04.921610 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:05.313762 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:05.313677 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:50:05.987534 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:05.987494 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8"] Jul 02 19:50:05.987998 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:05.987820 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="60bf7063-edea-4442-bde0-ddbb3ff645d6" containerName="istio-proxy" Jul 02 19:50:05.989689 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:05.989673 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" Jul 02 19:50:05.992345 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:05.992308 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-1d5ad8-serving-cert\"" Jul 02 19:50:05.992483 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:05.992429 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-wnsfc\"" Jul 02 19:50:06.003057 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:06.003029 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8"] Jul 02 19:50:06.111076 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:06.111040 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07c46638-5e2a-4709-b963-2e6596ff50c7-maas-api-tls\") pod \"maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8\" (UID: \"07c46638-5e2a-4709-b963-2e6596ff50c7\") " pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" Jul 02 19:50:06.111076 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:06.111082 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9m7s\" (UniqueName: \"kubernetes.io/projected/07c46638-5e2a-4709-b963-2e6596ff50c7-kube-api-access-r9m7s\") pod \"maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8\" (UID: \"07c46638-5e2a-4709-b963-2e6596ff50c7\") " pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" Jul 02 19:50:06.111311 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:06.111196 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07c46638-5e2a-4709-b963-2e6596ff50c7-openshift-service-ca\") pod \"maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8\" (UID: \"07c46638-5e2a-4709-b963-2e6596ff50c7\") " pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" Jul 02 19:50:06.212604 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:06.212569 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07c46638-5e2a-4709-b963-2e6596ff50c7-openshift-service-ca\") pod \"maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8\" (UID: \"07c46638-5e2a-4709-b963-2e6596ff50c7\") " pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" Jul 02 19:50:06.212788 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:06.212626 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07c46638-5e2a-4709-b963-2e6596ff50c7-maas-api-tls\") pod \"maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8\" (UID: \"07c46638-5e2a-4709-b963-2e6596ff50c7\") " pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" Jul 02 19:50:06.212788 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:06.212663 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9m7s\" (UniqueName: \"kubernetes.io/projected/07c46638-5e2a-4709-b963-2e6596ff50c7-kube-api-access-r9m7s\") pod \"maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8\" (UID: \"07c46638-5e2a-4709-b963-2e6596ff50c7\") " pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" Jul 02 19:50:06.213347 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:06.213322 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07c46638-5e2a-4709-b963-2e6596ff50c7-openshift-service-ca\") pod \"maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8\" (UID: \"07c46638-5e2a-4709-b963-2e6596ff50c7\") " pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" Jul 02 19:50:06.215322 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:06.215291 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07c46638-5e2a-4709-b963-2e6596ff50c7-maas-api-tls\") pod \"maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8\" (UID: \"07c46638-5e2a-4709-b963-2e6596ff50c7\") " pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" Jul 02 19:50:06.222586 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:06.222558 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9m7s\" (UniqueName: \"kubernetes.io/projected/07c46638-5e2a-4709-b963-2e6596ff50c7-kube-api-access-r9m7s\") pod \"maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8\" (UID: \"07c46638-5e2a-4709-b963-2e6596ff50c7\") " pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" Jul 02 19:50:06.301123 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:06.301020 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" Jul 02 19:50:06.437915 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:06.437881 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8"] Jul 02 19:50:06.441171 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:50:06.441133 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07c46638_5e2a_4709_b963_2e6596ff50c7.slice/crio-1f49564c6cb1120659790cdff70850c6ca5b378fe63dc83888c0b0525030b56d WatchSource:0}: Error finding container 1f49564c6cb1120659790cdff70850c6ca5b378fe63dc83888c0b0525030b56d: Status 404 returned error can't find the container with id 1f49564c6cb1120659790cdff70850c6ca5b378fe63dc83888c0b0525030b56d Jul 02 19:50:06.932004 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:06.931965 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" event={"ID":"07c46638-5e2a-4709-b963-2e6596ff50c7","Type":"ContainerStarted","Data":"7321593085dd1cf8eed2390dc25c6587be2d740bd8ecf95f8ef842d05697cafa"} Jul 02 19:50:06.932202 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:06.932030 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" event={"ID":"07c46638-5e2a-4709-b963-2e6596ff50c7","Type":"ContainerStarted","Data":"1f49564c6cb1120659790cdff70850c6ca5b378fe63dc83888c0b0525030b56d"} Jul 02 19:50:06.949670 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:06.949617 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" podStartSLOduration=1.549305614 podStartE2EDuration="1.949598583s" totalImagesPullingTime="400.292969ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:50:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:50:06.949027576 +0000 UTC m=+2319.495752604" watchObservedRunningTime="2026-07-02 19:50:06.949598583 +0000 UTC m=+2319.496323590" Jul 02 19:50:07.937162 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:07.937127 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" Jul 02 19:50:12.921311 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:12.921268 2556 patch_prober.go:28] interesting pod/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 02 19:50:12.921691 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:12.921337 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" podUID="f935e321-f1e8-4f4e-971f-5693823afdbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 02 19:50:13.947167 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:13.947123 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" Jul 02 19:50:13.966729 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:13.966699 2556 generic.go:356] "Generic (PLEG): container finished" podID="f935e321-f1e8-4f4e-971f-5693823afdbf" containerID="7afdad8037b0ade7e37f245b2cd5560c256cecb1eaa5630e1e57d28627a33f49" exitCode=0 Jul 02 19:50:13.966878 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:13.966771 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" event={"ID":"f935e321-f1e8-4f4e-971f-5693823afdbf","Type":"ContainerDied","Data":"7afdad8037b0ade7e37f245b2cd5560c256cecb1eaa5630e1e57d28627a33f49"} Jul 02 19:50:13.967185 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:13.967169 2556 scope.go:117] "RemoveContainer" containerID="7afdad8037b0ade7e37f245b2cd5560c256cecb1eaa5630e1e57d28627a33f49" Jul 02 19:50:13.968163 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:13.968116 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:13.968240 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:13.968221 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:13.968278 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:13.968267 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:14.973803 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:14.973762 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" event={"ID":"f935e321-f1e8-4f4e-971f-5693823afdbf","Type":"ContainerStarted","Data":"c8b40b5c9c87d476a55c88f0b7c2f533552fcf964305305a32541d7f825a9205"} Jul 02 19:50:15.050630 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:15.050545 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:50:15.050923 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:15.050900 2556 patch_prober.go:28] interesting pod/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 02 19:50:15.051055 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:15.051034 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" podUID="f935e321-f1e8-4f4e-971f-5693823afdbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 02 19:50:16.050458 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:16.050420 2556 patch_prober.go:28] interesting pod/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 02 19:50:16.050850 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:16.050480 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" podUID="f935e321-f1e8-4f4e-971f-5693823afdbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 02 19:50:17.049922 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:17.049882 2556 patch_prober.go:28] interesting pod/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 02 19:50:17.050138 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:17.049935 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" podUID="f935e321-f1e8-4f4e-971f-5693823afdbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 02 19:50:18.050844 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:18.050796 2556 patch_prober.go:28] interesting pod/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 02 19:50:18.051336 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:18.050856 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" podUID="f935e321-f1e8-4f4e-971f-5693823afdbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 02 19:50:19.050166 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:19.050110 2556 patch_prober.go:28] interesting pod/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 02 19:50:19.050400 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:19.050184 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" podUID="f935e321-f1e8-4f4e-971f-5693823afdbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 02 19:50:20.054605 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:20.054561 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:50:20.055040 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:20.054880 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:50:20.055785 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:20.055768 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:50:20.104312 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:20.104272 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:50:20.961310 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:20.961271 2556 patch_prober.go:28] interesting pod/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 02 19:50:20.965271 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:20.961348 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" podUID="290ec0dc-cfc7-46c8-a865-17f68f47701e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 02 19:50:22.013825 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:22.013791 2556 generic.go:356] "Generic (PLEG): container finished" podID="290ec0dc-cfc7-46c8-a865-17f68f47701e" containerID="9bab85cfc451558f6daef564d2ff136f23ae01525ce54a42ed64c6ebfe35aa73" exitCode=0 Jul 02 19:50:22.014240 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:22.013870 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" event={"ID":"290ec0dc-cfc7-46c8-a865-17f68f47701e","Type":"ContainerDied","Data":"9bab85cfc451558f6daef564d2ff136f23ae01525ce54a42ed64c6ebfe35aa73"} Jul 02 19:50:22.014396 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:22.014378 2556 scope.go:117] "RemoveContainer" containerID="9bab85cfc451558f6daef564d2ff136f23ae01525ce54a42ed64c6ebfe35aa73" Jul 02 19:50:22.015329 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:22.015303 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:22.015389 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:22.015369 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:22.015426 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:22.015402 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:23.022310 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:23.022272 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" event={"ID":"290ec0dc-cfc7-46c8-a865-17f68f47701e","Type":"ContainerStarted","Data":"b765fdd204e0f55c796a92d69b742661e1be54624428a28caa010c316715cdff"} Jul 02 19:50:23.433891 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:23.433852 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:23.434115 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:23.434091 2556 patch_prober.go:28] interesting pod/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 02 19:50:23.434195 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:23.434165 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" podUID="290ec0dc-cfc7-46c8-a865-17f68f47701e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 02 19:50:24.434278 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:24.434237 2556 patch_prober.go:28] interesting pod/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 02 19:50:24.434670 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:24.434299 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" podUID="290ec0dc-cfc7-46c8-a865-17f68f47701e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 02 19:50:24.727524 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:24.727438 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx"] Jul 02 19:50:24.727703 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:24.727656 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" podUID="f935e321-f1e8-4f4e-971f-5693823afdbf" containerName="istio-proxy" containerID="cri-o://c8b40b5c9c87d476a55c88f0b7c2f533552fcf964305305a32541d7f825a9205" gracePeriod=30 Jul 02 19:50:25.434595 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:25.434556 2556 patch_prober.go:28] interesting pod/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 02 19:50:25.435002 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:25.434619 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" podUID="290ec0dc-cfc7-46c8-a865-17f68f47701e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 02 19:50:26.222343 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:26.222303 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8"] Jul 02 19:50:26.222576 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:26.222546 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" podUID="07c46638-5e2a-4709-b963-2e6596ff50c7" containerName="maas-api" containerID="cri-o://7321593085dd1cf8eed2390dc25c6587be2d740bd8ecf95f8ef842d05697cafa" gracePeriod=30 Jul 02 19:50:26.433939 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:26.433911 2556 patch_prober.go:28] interesting pod/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 02 19:50:26.434336 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:26.434228 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" podUID="290ec0dc-cfc7-46c8-a865-17f68f47701e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 02 19:50:26.482041 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:26.481982 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" Jul 02 19:50:26.585340 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:26.585303 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9m7s\" (UniqueName: \"kubernetes.io/projected/07c46638-5e2a-4709-b963-2e6596ff50c7-kube-api-access-r9m7s\") pod \"07c46638-5e2a-4709-b963-2e6596ff50c7\" (UID: \"07c46638-5e2a-4709-b963-2e6596ff50c7\") " Jul 02 19:50:26.585340 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:26.585353 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07c46638-5e2a-4709-b963-2e6596ff50c7-openshift-service-ca\") pod \"07c46638-5e2a-4709-b963-2e6596ff50c7\" (UID: \"07c46638-5e2a-4709-b963-2e6596ff50c7\") " Jul 02 19:50:26.585670 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:26.585383 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07c46638-5e2a-4709-b963-2e6596ff50c7-maas-api-tls\") pod \"07c46638-5e2a-4709-b963-2e6596ff50c7\" (UID: \"07c46638-5e2a-4709-b963-2e6596ff50c7\") " Jul 02 19:50:26.585746 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:26.585715 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07c46638-5e2a-4709-b963-2e6596ff50c7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "07c46638-5e2a-4709-b963-2e6596ff50c7" (UID: "07c46638-5e2a-4709-b963-2e6596ff50c7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:50:26.587406 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:26.587375 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07c46638-5e2a-4709-b963-2e6596ff50c7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "07c46638-5e2a-4709-b963-2e6596ff50c7" (UID: "07c46638-5e2a-4709-b963-2e6596ff50c7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:50:26.587512 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:26.587474 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07c46638-5e2a-4709-b963-2e6596ff50c7-kube-api-access-r9m7s" (OuterVolumeSpecName: "kube-api-access-r9m7s") pod "07c46638-5e2a-4709-b963-2e6596ff50c7" (UID: "07c46638-5e2a-4709-b963-2e6596ff50c7"). InnerVolumeSpecName "kube-api-access-r9m7s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:50:26.686063 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:26.686023 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9m7s\" (UniqueName: \"kubernetes.io/projected/07c46638-5e2a-4709-b963-2e6596ff50c7-kube-api-access-r9m7s\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:26.686063 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:26.686056 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07c46638-5e2a-4709-b963-2e6596ff50c7-openshift-service-ca\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:26.686063 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:26.686066 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07c46638-5e2a-4709-b963-2e6596ff50c7-maas-api-tls\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:27.042894 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:27.042861 2556 generic.go:356] "Generic (PLEG): container finished" podID="07c46638-5e2a-4709-b963-2e6596ff50c7" containerID="7321593085dd1cf8eed2390dc25c6587be2d740bd8ecf95f8ef842d05697cafa" exitCode=0 Jul 02 19:50:27.043063 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:27.042908 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" Jul 02 19:50:27.043063 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:27.042946 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" event={"ID":"07c46638-5e2a-4709-b963-2e6596ff50c7","Type":"ContainerDied","Data":"7321593085dd1cf8eed2390dc25c6587be2d740bd8ecf95f8ef842d05697cafa"} Jul 02 19:50:27.043063 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:27.043000 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8" event={"ID":"07c46638-5e2a-4709-b963-2e6596ff50c7","Type":"ContainerDied","Data":"1f49564c6cb1120659790cdff70850c6ca5b378fe63dc83888c0b0525030b56d"} Jul 02 19:50:27.043063 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:27.043025 2556 scope.go:117] "RemoveContainer" containerID="7321593085dd1cf8eed2390dc25c6587be2d740bd8ecf95f8ef842d05697cafa" Jul 02 19:50:27.052293 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:27.052276 2556 scope.go:117] "RemoveContainer" containerID="7321593085dd1cf8eed2390dc25c6587be2d740bd8ecf95f8ef842d05697cafa" Jul 02 19:50:27.052551 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:50:27.052530 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7321593085dd1cf8eed2390dc25c6587be2d740bd8ecf95f8ef842d05697cafa\": container with ID starting with 7321593085dd1cf8eed2390dc25c6587be2d740bd8ecf95f8ef842d05697cafa not found: ID does not exist" containerID="7321593085dd1cf8eed2390dc25c6587be2d740bd8ecf95f8ef842d05697cafa" Jul 02 19:50:27.052592 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:27.052560 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7321593085dd1cf8eed2390dc25c6587be2d740bd8ecf95f8ef842d05697cafa"} err="failed to get container status \"7321593085dd1cf8eed2390dc25c6587be2d740bd8ecf95f8ef842d05697cafa\": rpc error: code = NotFound desc = could not find container \"7321593085dd1cf8eed2390dc25c6587be2d740bd8ecf95f8ef842d05697cafa\": container with ID starting with 7321593085dd1cf8eed2390dc25c6587be2d740bd8ecf95f8ef842d05697cafa not found: ID does not exist" Jul 02 19:50:27.067614 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:27.067580 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8"] Jul 02 19:50:27.069883 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:27.069863 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-1d5ad8-6d85c87f45-h2ng8"] Jul 02 19:50:27.437136 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:27.437105 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:27.437406 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:27.437389 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:27.438009 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:27.437993 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:27.978543 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:27.978512 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07c46638-5e2a-4709-b963-2e6596ff50c7" path="/var/lib/kubelet/pods/07c46638-5e2a-4709-b963-2e6596ff50c7/volumes" Jul 02 19:50:29.698011 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:29.697970 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:50:29.969044 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:29.969019 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:50:30.015315 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.015276 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-data\") pod \"f935e321-f1e8-4f4e-971f-5693823afdbf\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " Jul 02 19:50:30.015499 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.015327 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-workload-socket\") pod \"f935e321-f1e8-4f4e-971f-5693823afdbf\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " Jul 02 19:50:30.015499 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.015368 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-envoy\") pod \"f935e321-f1e8-4f4e-971f-5693823afdbf\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " Jul 02 19:50:30.015499 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.015406 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-token\") pod \"f935e321-f1e8-4f4e-971f-5693823afdbf\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " Jul 02 19:50:30.015499 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.015435 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xztt7\" (UniqueName: \"kubernetes.io/projected/f935e321-f1e8-4f4e-971f-5693823afdbf-kube-api-access-xztt7\") pod \"f935e321-f1e8-4f4e-971f-5693823afdbf\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " Jul 02 19:50:30.015499 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.015496 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-podinfo\") pod \"f935e321-f1e8-4f4e-971f-5693823afdbf\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " Jul 02 19:50:30.015768 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.015559 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f935e321-f1e8-4f4e-971f-5693823afdbf-istiod-ca-cert\") pod \"f935e321-f1e8-4f4e-971f-5693823afdbf\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " Jul 02 19:50:30.015768 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.015588 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-credential-socket\") pod \"f935e321-f1e8-4f4e-971f-5693823afdbf\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " Jul 02 19:50:30.015997 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.015945 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f935e321-f1e8-4f4e-971f-5693823afdbf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f935e321-f1e8-4f4e-971f-5693823afdbf" (UID: "f935e321-f1e8-4f4e-971f-5693823afdbf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:50:30.016424 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.015976 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f935e321-f1e8-4f4e-971f-5693823afdbf" (UID: "f935e321-f1e8-4f4e-971f-5693823afdbf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:50:30.016424 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.016108 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-workload-certs\") pod \"f935e321-f1e8-4f4e-971f-5693823afdbf\" (UID: \"f935e321-f1e8-4f4e-971f-5693823afdbf\") " Jul 02 19:50:30.016424 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.016385 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-workload-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:30.016424 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.016407 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f935e321-f1e8-4f4e-971f-5693823afdbf-istiod-ca-cert\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:30.016746 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.016484 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-data" (OuterVolumeSpecName: "istio-data") pod "f935e321-f1e8-4f4e-971f-5693823afdbf" (UID: "f935e321-f1e8-4f4e-971f-5693823afdbf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:50:30.016746 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.016684 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f935e321-f1e8-4f4e-971f-5693823afdbf" (UID: "f935e321-f1e8-4f4e-971f-5693823afdbf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:50:30.016746 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.016704 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f935e321-f1e8-4f4e-971f-5693823afdbf" (UID: "f935e321-f1e8-4f4e-971f-5693823afdbf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:50:30.017885 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.017857 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f935e321-f1e8-4f4e-971f-5693823afdbf" (UID: "f935e321-f1e8-4f4e-971f-5693823afdbf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:50:30.018238 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.018215 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f935e321-f1e8-4f4e-971f-5693823afdbf" (UID: "f935e321-f1e8-4f4e-971f-5693823afdbf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:50:30.018323 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.018230 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f935e321-f1e8-4f4e-971f-5693823afdbf-kube-api-access-xztt7" (OuterVolumeSpecName: "kube-api-access-xztt7") pod "f935e321-f1e8-4f4e-971f-5693823afdbf" (UID: "f935e321-f1e8-4f4e-971f-5693823afdbf"). InnerVolumeSpecName "kube-api-access-xztt7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:50:30.018323 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.018234 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-token" (OuterVolumeSpecName: "istio-token") pod "f935e321-f1e8-4f4e-971f-5693823afdbf" (UID: "f935e321-f1e8-4f4e-971f-5693823afdbf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:50:30.065539 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.065502 2556 generic.go:356] "Generic (PLEG): container finished" podID="f935e321-f1e8-4f4e-971f-5693823afdbf" containerID="c8b40b5c9c87d476a55c88f0b7c2f533552fcf964305305a32541d7f825a9205" exitCode=0 Jul 02 19:50:30.065696 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.065564 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" Jul 02 19:50:30.065696 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.065584 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" event={"ID":"f935e321-f1e8-4f4e-971f-5693823afdbf","Type":"ContainerDied","Data":"c8b40b5c9c87d476a55c88f0b7c2f533552fcf964305305a32541d7f825a9205"} Jul 02 19:50:30.065696 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.065642 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx" event={"ID":"f935e321-f1e8-4f4e-971f-5693823afdbf","Type":"ContainerDied","Data":"5f3ad7e46c6bcad864ae27ad11d7ae87961fb65fda05be6790ed9b634d1e6d0f"} Jul 02 19:50:30.065696 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.065665 2556 scope.go:117] "RemoveContainer" containerID="c8b40b5c9c87d476a55c88f0b7c2f533552fcf964305305a32541d7f825a9205" Jul 02 19:50:30.075291 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.075274 2556 scope.go:117] "RemoveContainer" containerID="7afdad8037b0ade7e37f245b2cd5560c256cecb1eaa5630e1e57d28627a33f49" Jul 02 19:50:30.083396 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.083378 2556 scope.go:117] "RemoveContainer" containerID="c8b40b5c9c87d476a55c88f0b7c2f533552fcf964305305a32541d7f825a9205" Jul 02 19:50:30.083639 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:50:30.083620 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8b40b5c9c87d476a55c88f0b7c2f533552fcf964305305a32541d7f825a9205\": container with ID starting with c8b40b5c9c87d476a55c88f0b7c2f533552fcf964305305a32541d7f825a9205 not found: ID does not exist" containerID="c8b40b5c9c87d476a55c88f0b7c2f533552fcf964305305a32541d7f825a9205" Jul 02 19:50:30.083686 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.083649 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8b40b5c9c87d476a55c88f0b7c2f533552fcf964305305a32541d7f825a9205"} err="failed to get container status \"c8b40b5c9c87d476a55c88f0b7c2f533552fcf964305305a32541d7f825a9205\": rpc error: code = NotFound desc = could not find container \"c8b40b5c9c87d476a55c88f0b7c2f533552fcf964305305a32541d7f825a9205\": container with ID starting with c8b40b5c9c87d476a55c88f0b7c2f533552fcf964305305a32541d7f825a9205 not found: ID does not exist" Jul 02 19:50:30.083686 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.083666 2556 scope.go:117] "RemoveContainer" containerID="7afdad8037b0ade7e37f245b2cd5560c256cecb1eaa5630e1e57d28627a33f49" Jul 02 19:50:30.083907 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:50:30.083888 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7afdad8037b0ade7e37f245b2cd5560c256cecb1eaa5630e1e57d28627a33f49\": container with ID starting with 7afdad8037b0ade7e37f245b2cd5560c256cecb1eaa5630e1e57d28627a33f49 not found: ID does not exist" containerID="7afdad8037b0ade7e37f245b2cd5560c256cecb1eaa5630e1e57d28627a33f49" Jul 02 19:50:30.083962 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.083916 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7afdad8037b0ade7e37f245b2cd5560c256cecb1eaa5630e1e57d28627a33f49"} err="failed to get container status \"7afdad8037b0ade7e37f245b2cd5560c256cecb1eaa5630e1e57d28627a33f49\": rpc error: code = NotFound desc = could not find container \"7afdad8037b0ade7e37f245b2cd5560c256cecb1eaa5630e1e57d28627a33f49\": container with ID starting with 7afdad8037b0ade7e37f245b2cd5560c256cecb1eaa5630e1e57d28627a33f49 not found: ID does not exist" Jul 02 19:50:30.091351 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.091329 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx"] Jul 02 19:50:30.095605 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.095587 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-455cd8-openshift-default-58fcb6c985-xbkjx"] Jul 02 19:50:30.116971 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.116946 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-data\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:30.116971 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.116969 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-envoy\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:30.116971 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.116977 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-token\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:30.117177 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.116986 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xztt7\" (UniqueName: \"kubernetes.io/projected/f935e321-f1e8-4f4e-971f-5693823afdbf-kube-api-access-xztt7\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:30.117177 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.116996 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f935e321-f1e8-4f4e-971f-5693823afdbf-istio-podinfo\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:30.117177 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.117005 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-credential-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:30.117177 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:30.117013 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f935e321-f1e8-4f4e-971f-5693823afdbf-workload-certs\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:31.710426 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:31.710389 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5"] Jul 02 19:50:31.710853 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:31.710631 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" podUID="290ec0dc-cfc7-46c8-a865-17f68f47701e" containerName="istio-proxy" containerID="cri-o://b765fdd204e0f55c796a92d69b742661e1be54624428a28caa010c316715cdff" gracePeriod=30 Jul 02 19:50:31.977793 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:31.977705 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f935e321-f1e8-4f4e-971f-5693823afdbf" path="/var/lib/kubelet/pods/f935e321-f1e8-4f4e-971f-5693823afdbf/volumes" Jul 02 19:50:32.542790 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.542758 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv"] Jul 02 19:50:32.543070 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.543059 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f935e321-f1e8-4f4e-971f-5693823afdbf" containerName="istio-proxy" Jul 02 19:50:32.543118 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.543072 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f935e321-f1e8-4f4e-971f-5693823afdbf" containerName="istio-proxy" Jul 02 19:50:32.543118 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.543099 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07c46638-5e2a-4709-b963-2e6596ff50c7" containerName="maas-api" Jul 02 19:50:32.543118 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.543105 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="07c46638-5e2a-4709-b963-2e6596ff50c7" containerName="maas-api" Jul 02 19:50:32.543238 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.543166 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f935e321-f1e8-4f4e-971f-5693823afdbf" containerName="istio-proxy" Jul 02 19:50:32.543238 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.543183 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="07c46638-5e2a-4709-b963-2e6596ff50c7" containerName="maas-api" Jul 02 19:50:32.547556 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.547533 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.550686 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.550664 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-2fca4d-openshift-default-dockercfg-bxp47\"" Jul 02 19:50:32.565758 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.565727 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv"] Jul 02 19:50:32.637325 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.637281 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-token\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.637325 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.637325 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-podinfo\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.637581 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.637353 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-envoy\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.637581 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.637406 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istiod-ca-cert\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.637581 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.637429 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-workload-certs\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.637581 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.637460 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-workload-socket\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.637581 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.637488 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqgd5\" (UniqueName: \"kubernetes.io/projected/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-kube-api-access-xqgd5\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.637581 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.637516 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-data\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.637581 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.637538 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-credential-socket\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.738695 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.738656 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istiod-ca-cert\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.738695 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.738698 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-workload-certs\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.739280 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.738732 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-workload-socket\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.739280 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.738773 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqgd5\" (UniqueName: \"kubernetes.io/projected/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-kube-api-access-xqgd5\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.739280 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.738802 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-data\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.739280 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.738829 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-credential-socket\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.739280 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.738879 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-token\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.739280 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.738908 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-podinfo\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.739280 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.738938 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-envoy\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.739280 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.739198 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-workload-certs\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.739535 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.739313 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-credential-socket\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.739535 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.739337 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-data\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.739535 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.739450 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-workload-socket\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.739629 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.739559 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istiod-ca-cert\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.741369 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.741347 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-envoy\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.741714 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.741695 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-podinfo\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.749404 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.749377 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-token\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.749765 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.749744 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqgd5\" (UniqueName: \"kubernetes.io/projected/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-kube-api-access-xqgd5\") pod \"e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.862971 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.862882 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:32.998052 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:32.998026 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv"] Jul 02 19:50:32.999468 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:50:32.999442 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7f3aeb8_3c79_4fc7_be42_37bd18ef682c.slice/crio-c195ea58fb251ee5390e8fad3309a2b7b9da9c5b41e291e6f9bee4e4a2ead442 WatchSource:0}: Error finding container c195ea58fb251ee5390e8fad3309a2b7b9da9c5b41e291e6f9bee4e4a2ead442: Status 404 returned error can't find the container with id c195ea58fb251ee5390e8fad3309a2b7b9da9c5b41e291e6f9bee4e4a2ead442 Jul 02 19:50:33.001700 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:33.001668 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:33.001792 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:33.001734 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:33.001792 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:33.001762 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:33.082140 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:33.082101 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" event={"ID":"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c","Type":"ContainerStarted","Data":"9f4cc9ebec07ec768b7075fd666ac70cc61df8e2bee5bc302908bd87013759f1"} Jul 02 19:50:33.082309 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:33.082180 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" event={"ID":"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c","Type":"ContainerStarted","Data":"c195ea58fb251ee5390e8fad3309a2b7b9da9c5b41e291e6f9bee4e4a2ead442"} Jul 02 19:50:33.107198 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:33.107130 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" podStartSLOduration=1.107114061 podStartE2EDuration="1.107114061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:50:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:50:33.105675924 +0000 UTC m=+2345.652400929" watchObservedRunningTime="2026-07-02 19:50:33.107114061 +0000 UTC m=+2345.653839068" Jul 02 19:50:33.863307 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:33.863272 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:34.868655 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:34.868626 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:35.092819 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.092792 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:35.094063 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.094033 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:35.164791 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.164682 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx"] Jul 02 19:50:35.165107 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.165079 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f935e321-f1e8-4f4e-971f-5693823afdbf" containerName="istio-proxy" Jul 02 19:50:35.165230 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.165109 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f935e321-f1e8-4f4e-971f-5693823afdbf" containerName="istio-proxy" Jul 02 19:50:35.165230 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.165206 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f935e321-f1e8-4f4e-971f-5693823afdbf" containerName="istio-proxy" Jul 02 19:50:35.168054 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.168031 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" Jul 02 19:50:35.170716 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.170694 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-wnsfc\"" Jul 02 19:50:35.171000 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.170717 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-2fca4d-serving-cert\"" Jul 02 19:50:35.181218 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.181194 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx"] Jul 02 19:50:35.263405 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.263334 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/837919fb-335a-46ed-85ea-0cb88b508dcc-maas-api-tls\") pod \"maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx\" (UID: \"837919fb-335a-46ed-85ea-0cb88b508dcc\") " pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" Jul 02 19:50:35.263648 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.263621 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/837919fb-335a-46ed-85ea-0cb88b508dcc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx\" (UID: \"837919fb-335a-46ed-85ea-0cb88b508dcc\") " pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" Jul 02 19:50:35.263793 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.263659 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp5tk\" (UniqueName: \"kubernetes.io/projected/837919fb-335a-46ed-85ea-0cb88b508dcc-kube-api-access-dp5tk\") pod \"maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx\" (UID: \"837919fb-335a-46ed-85ea-0cb88b508dcc\") " pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" Jul 02 19:50:35.364743 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.364697 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/837919fb-335a-46ed-85ea-0cb88b508dcc-maas-api-tls\") pod \"maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx\" (UID: \"837919fb-335a-46ed-85ea-0cb88b508dcc\") " pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" Jul 02 19:50:35.364743 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.364744 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/837919fb-335a-46ed-85ea-0cb88b508dcc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx\" (UID: \"837919fb-335a-46ed-85ea-0cb88b508dcc\") " pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" Jul 02 19:50:35.365013 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.364768 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp5tk\" (UniqueName: \"kubernetes.io/projected/837919fb-335a-46ed-85ea-0cb88b508dcc-kube-api-access-dp5tk\") pod \"maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx\" (UID: \"837919fb-335a-46ed-85ea-0cb88b508dcc\") " pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" Jul 02 19:50:35.365013 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:50:35.364861 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-2fca4d-serving-cert: secret "maas-api-e2e-shared-a-2fca4d-serving-cert" not found Jul 02 19:50:35.365013 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:50:35.364940 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/837919fb-335a-46ed-85ea-0cb88b508dcc-maas-api-tls podName:837919fb-335a-46ed-85ea-0cb88b508dcc nodeName:}" failed. No retries permitted until 2026-07-02 19:50:35.864920169 +0000 UTC m=+2348.411645155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/837919fb-335a-46ed-85ea-0cb88b508dcc-maas-api-tls") pod "maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" (UID: "837919fb-335a-46ed-85ea-0cb88b508dcc") : secret "maas-api-e2e-shared-a-2fca4d-serving-cert" not found Jul 02 19:50:35.365448 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.365429 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/837919fb-335a-46ed-85ea-0cb88b508dcc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx\" (UID: \"837919fb-335a-46ed-85ea-0cb88b508dcc\") " pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" Jul 02 19:50:35.374210 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.374191 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp5tk\" (UniqueName: \"kubernetes.io/projected/837919fb-335a-46ed-85ea-0cb88b508dcc-kube-api-access-dp5tk\") pod \"maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx\" (UID: \"837919fb-335a-46ed-85ea-0cb88b508dcc\") " pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" Jul 02 19:50:35.869476 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.869434 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/837919fb-335a-46ed-85ea-0cb88b508dcc-maas-api-tls\") pod \"maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx\" (UID: \"837919fb-335a-46ed-85ea-0cb88b508dcc\") " pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" Jul 02 19:50:35.871798 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:35.871778 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/837919fb-335a-46ed-85ea-0cb88b508dcc-maas-api-tls\") pod \"maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx\" (UID: \"837919fb-335a-46ed-85ea-0cb88b508dcc\") " pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" Jul 02 19:50:36.081984 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:36.081944 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" Jul 02 19:50:36.216771 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:36.216743 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx"] Jul 02 19:50:36.217910 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:50:36.217882 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod837919fb_335a_46ed_85ea_0cb88b508dcc.slice/crio-f26314266e6b59bebb87903f22ac706dd7f1a6fe32fa3048e5f1b9a125bf5ad5 WatchSource:0}: Error finding container f26314266e6b59bebb87903f22ac706dd7f1a6fe32fa3048e5f1b9a125bf5ad5: Status 404 returned error can't find the container with id f26314266e6b59bebb87903f22ac706dd7f1a6fe32fa3048e5f1b9a125bf5ad5 Jul 02 19:50:36.960961 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:36.960934 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:37.079979 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.079877 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-credential-socket\") pod \"290ec0dc-cfc7-46c8-a865-17f68f47701e\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " Jul 02 19:50:37.079979 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.079918 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-envoy\") pod \"290ec0dc-cfc7-46c8-a865-17f68f47701e\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " Jul 02 19:50:37.079979 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.079946 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2q55\" (UniqueName: \"kubernetes.io/projected/290ec0dc-cfc7-46c8-a865-17f68f47701e-kube-api-access-n2q55\") pod \"290ec0dc-cfc7-46c8-a865-17f68f47701e\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " Jul 02 19:50:37.079979 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.079967 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-workload-socket\") pod \"290ec0dc-cfc7-46c8-a865-17f68f47701e\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " Jul 02 19:50:37.080339 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.079991 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-workload-certs\") pod \"290ec0dc-cfc7-46c8-a865-17f68f47701e\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " Jul 02 19:50:37.080339 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.080022 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-podinfo\") pod \"290ec0dc-cfc7-46c8-a865-17f68f47701e\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " Jul 02 19:50:37.080339 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.080049 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/290ec0dc-cfc7-46c8-a865-17f68f47701e-istiod-ca-cert\") pod \"290ec0dc-cfc7-46c8-a865-17f68f47701e\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " Jul 02 19:50:37.080339 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.080091 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-token\") pod \"290ec0dc-cfc7-46c8-a865-17f68f47701e\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " Jul 02 19:50:37.080339 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.080114 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-data\") pod \"290ec0dc-cfc7-46c8-a865-17f68f47701e\" (UID: \"290ec0dc-cfc7-46c8-a865-17f68f47701e\") " Jul 02 19:50:37.080339 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.080246 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "290ec0dc-cfc7-46c8-a865-17f68f47701e" (UID: "290ec0dc-cfc7-46c8-a865-17f68f47701e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:50:37.080640 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.080402 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-credential-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:37.080640 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.080435 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "290ec0dc-cfc7-46c8-a865-17f68f47701e" (UID: "290ec0dc-cfc7-46c8-a865-17f68f47701e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:50:37.080640 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.080533 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/290ec0dc-cfc7-46c8-a865-17f68f47701e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "290ec0dc-cfc7-46c8-a865-17f68f47701e" (UID: "290ec0dc-cfc7-46c8-a865-17f68f47701e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:50:37.080754 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.080643 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "290ec0dc-cfc7-46c8-a865-17f68f47701e" (UID: "290ec0dc-cfc7-46c8-a865-17f68f47701e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:50:37.081224 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.081201 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-data" (OuterVolumeSpecName: "istio-data") pod "290ec0dc-cfc7-46c8-a865-17f68f47701e" (UID: "290ec0dc-cfc7-46c8-a865-17f68f47701e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:50:37.082219 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.082189 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "290ec0dc-cfc7-46c8-a865-17f68f47701e" (UID: "290ec0dc-cfc7-46c8-a865-17f68f47701e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:50:37.082389 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.082375 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-token" (OuterVolumeSpecName: "istio-token") pod "290ec0dc-cfc7-46c8-a865-17f68f47701e" (UID: "290ec0dc-cfc7-46c8-a865-17f68f47701e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:50:37.082736 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.082710 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "290ec0dc-cfc7-46c8-a865-17f68f47701e" (UID: "290ec0dc-cfc7-46c8-a865-17f68f47701e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:50:37.082736 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.082712 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/290ec0dc-cfc7-46c8-a865-17f68f47701e-kube-api-access-n2q55" (OuterVolumeSpecName: "kube-api-access-n2q55") pod "290ec0dc-cfc7-46c8-a865-17f68f47701e" (UID: "290ec0dc-cfc7-46c8-a865-17f68f47701e"). InnerVolumeSpecName "kube-api-access-n2q55". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:50:37.104408 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.104364 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" event={"ID":"837919fb-335a-46ed-85ea-0cb88b508dcc","Type":"ContainerStarted","Data":"717fa1df17fdf19f47de15465296d8cdc1a817a4024707c33b757ff700b111a6"} Jul 02 19:50:37.104555 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.104437 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" event={"ID":"837919fb-335a-46ed-85ea-0cb88b508dcc","Type":"ContainerStarted","Data":"f26314266e6b59bebb87903f22ac706dd7f1a6fe32fa3048e5f1b9a125bf5ad5"} Jul 02 19:50:37.104555 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.104468 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" Jul 02 19:50:37.105902 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.105879 2556 generic.go:356] "Generic (PLEG): container finished" podID="290ec0dc-cfc7-46c8-a865-17f68f47701e" containerID="b765fdd204e0f55c796a92d69b742661e1be54624428a28caa010c316715cdff" exitCode=0 Jul 02 19:50:37.106015 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.105918 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" Jul 02 19:50:37.106015 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.105956 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" event={"ID":"290ec0dc-cfc7-46c8-a865-17f68f47701e","Type":"ContainerDied","Data":"b765fdd204e0f55c796a92d69b742661e1be54624428a28caa010c316715cdff"} Jul 02 19:50:37.106015 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.106000 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5" event={"ID":"290ec0dc-cfc7-46c8-a865-17f68f47701e","Type":"ContainerDied","Data":"10ba649acf925f6851e59a754952cb6e86a6bf3a93bc347647724ef962d709e8"} Jul 02 19:50:37.106129 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.106024 2556 scope.go:117] "RemoveContainer" containerID="b765fdd204e0f55c796a92d69b742661e1be54624428a28caa010c316715cdff" Jul 02 19:50:37.116470 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.116449 2556 scope.go:117] "RemoveContainer" containerID="9bab85cfc451558f6daef564d2ff136f23ae01525ce54a42ed64c6ebfe35aa73" Jul 02 19:50:37.121521 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.121468 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" podStartSLOduration=1.788844848 podStartE2EDuration="2.121448597s" totalImagesPullingTime="332.603749ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:50:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:50:37.119900757 +0000 UTC m=+2349.666625767" watchObservedRunningTime="2026-07-02 19:50:37.121448597 +0000 UTC m=+2349.668173606" Jul 02 19:50:37.125727 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.125704 2556 scope.go:117] "RemoveContainer" containerID="b765fdd204e0f55c796a92d69b742661e1be54624428a28caa010c316715cdff" Jul 02 19:50:37.126206 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:50:37.126180 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b765fdd204e0f55c796a92d69b742661e1be54624428a28caa010c316715cdff\": container with ID starting with b765fdd204e0f55c796a92d69b742661e1be54624428a28caa010c316715cdff not found: ID does not exist" containerID="b765fdd204e0f55c796a92d69b742661e1be54624428a28caa010c316715cdff" Jul 02 19:50:37.126309 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.126213 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b765fdd204e0f55c796a92d69b742661e1be54624428a28caa010c316715cdff"} err="failed to get container status \"b765fdd204e0f55c796a92d69b742661e1be54624428a28caa010c316715cdff\": rpc error: code = NotFound desc = could not find container \"b765fdd204e0f55c796a92d69b742661e1be54624428a28caa010c316715cdff\": container with ID starting with b765fdd204e0f55c796a92d69b742661e1be54624428a28caa010c316715cdff not found: ID does not exist" Jul 02 19:50:37.126309 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.126233 2556 scope.go:117] "RemoveContainer" containerID="9bab85cfc451558f6daef564d2ff136f23ae01525ce54a42ed64c6ebfe35aa73" Jul 02 19:50:37.126592 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:50:37.126569 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bab85cfc451558f6daef564d2ff136f23ae01525ce54a42ed64c6ebfe35aa73\": container with ID starting with 9bab85cfc451558f6daef564d2ff136f23ae01525ce54a42ed64c6ebfe35aa73 not found: ID does not exist" containerID="9bab85cfc451558f6daef564d2ff136f23ae01525ce54a42ed64c6ebfe35aa73" Jul 02 19:50:37.126690 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.126600 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bab85cfc451558f6daef564d2ff136f23ae01525ce54a42ed64c6ebfe35aa73"} err="failed to get container status \"9bab85cfc451558f6daef564d2ff136f23ae01525ce54a42ed64c6ebfe35aa73\": rpc error: code = NotFound desc = could not find container \"9bab85cfc451558f6daef564d2ff136f23ae01525ce54a42ed64c6ebfe35aa73\": container with ID starting with 9bab85cfc451558f6daef564d2ff136f23ae01525ce54a42ed64c6ebfe35aa73 not found: ID does not exist" Jul 02 19:50:37.133489 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.133467 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5"] Jul 02 19:50:37.137847 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.137814 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-1d5ad8-openshift-default-696cf4b899-fqqh5"] Jul 02 19:50:37.181878 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.181835 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-envoy\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:37.182003 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.181935 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2q55\" (UniqueName: \"kubernetes.io/projected/290ec0dc-cfc7-46c8-a865-17f68f47701e-kube-api-access-n2q55\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:37.182003 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.181949 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-workload-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:37.182003 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.181959 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-workload-certs\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:37.182003 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.181968 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-podinfo\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:37.182003 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.181976 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/290ec0dc-cfc7-46c8-a865-17f68f47701e-istiod-ca-cert\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:37.182003 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.181983 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-token\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:37.182003 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.181992 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/290ec0dc-cfc7-46c8-a865-17f68f47701e-istio-data\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:50:37.978375 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:37.978335 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="290ec0dc-cfc7-46c8-a865-17f68f47701e" path="/var/lib/kubelet/pods/290ec0dc-cfc7-46c8-a865-17f68f47701e/volumes" Jul 02 19:50:40.172274 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.172241 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs"] Jul 02 19:50:40.172758 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.172701 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="290ec0dc-cfc7-46c8-a865-17f68f47701e" containerName="istio-proxy" Jul 02 19:50:40.172758 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.172722 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="290ec0dc-cfc7-46c8-a865-17f68f47701e" containerName="istio-proxy" Jul 02 19:50:40.172853 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.172825 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="290ec0dc-cfc7-46c8-a865-17f68f47701e" containerName="istio-proxy" Jul 02 19:50:40.177562 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.177543 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.180006 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.179987 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-f9d728-openshift-default-dockercfg-jdjc7\"" Jul 02 19:50:40.187778 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.187755 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs"] Jul 02 19:50:40.308376 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.308344 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4751daa8-9b94-4270-ac05-59f561afeefb-istio-token\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.308376 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.308381 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-workload-certs\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.308622 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.308420 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4751daa8-9b94-4270-ac05-59f561afeefb-istiod-ca-cert\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.308622 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.308438 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-istio-data\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.308622 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.308530 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-istio-envoy\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.308622 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.308574 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-workload-socket\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.308622 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.308609 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-credential-socket\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.308855 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.308669 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4751daa8-9b94-4270-ac05-59f561afeefb-istio-podinfo\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.308855 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.308695 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvhg5\" (UniqueName: \"kubernetes.io/projected/4751daa8-9b94-4270-ac05-59f561afeefb-kube-api-access-nvhg5\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.404737 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.404700 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:50:40.409404 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.409381 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4751daa8-9b94-4270-ac05-59f561afeefb-istio-podinfo\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.409489 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.409415 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvhg5\" (UniqueName: \"kubernetes.io/projected/4751daa8-9b94-4270-ac05-59f561afeefb-kube-api-access-nvhg5\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.409489 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.409458 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4751daa8-9b94-4270-ac05-59f561afeefb-istio-token\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.409602 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.409487 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-workload-certs\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.409602 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.409533 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4751daa8-9b94-4270-ac05-59f561afeefb-istiod-ca-cert\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.409701 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.409661 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-istio-data\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.409750 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.409716 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-istio-envoy\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.409813 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.409756 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-workload-socket\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.409813 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.409780 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-credential-socket\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.410000 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.409977 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-workload-certs\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.410070 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.410029 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-istio-data\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.410127 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.410080 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-workload-socket\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.410206 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.410135 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-credential-socket\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.410312 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.410292 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4751daa8-9b94-4270-ac05-59f561afeefb-istiod-ca-cert\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.411873 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.411844 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-istio-envoy\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.412008 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.411992 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4751daa8-9b94-4270-ac05-59f561afeefb-istio-podinfo\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.418386 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.418362 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4751daa8-9b94-4270-ac05-59f561afeefb-istio-token\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.418501 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.418420 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvhg5\" (UniqueName: \"kubernetes.io/projected/4751daa8-9b94-4270-ac05-59f561afeefb-kube-api-access-nvhg5\") pod \"e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.488348 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.488259 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:40.626826 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.626797 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs"] Jul 02 19:50:40.627928 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:50:40.627897 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4751daa8_9b94_4270_ac05_59f561afeefb.slice/crio-37ea281ea90b31cdfca6c29a87e074e67b829aa2d322f85ac8ccd09f9983ad71 WatchSource:0}: Error finding container 37ea281ea90b31cdfca6c29a87e074e67b829aa2d322f85ac8ccd09f9983ad71: Status 404 returned error can't find the container with id 37ea281ea90b31cdfca6c29a87e074e67b829aa2d322f85ac8ccd09f9983ad71 Jul 02 19:50:40.630546 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.630520 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:40.630622 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.630578 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:40.630622 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:40.630604 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:41.128668 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:41.128631 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" event={"ID":"4751daa8-9b94-4270-ac05-59f561afeefb","Type":"ContainerStarted","Data":"a8cd00fc1f27400a22e535c5b8acf5c3b08aab0f07eff5f730853451d4de8994"} Jul 02 19:50:41.128858 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:41.128688 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" event={"ID":"4751daa8-9b94-4270-ac05-59f561afeefb","Type":"ContainerStarted","Data":"37ea281ea90b31cdfca6c29a87e074e67b829aa2d322f85ac8ccd09f9983ad71"} Jul 02 19:50:41.147762 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:41.147711 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" podStartSLOduration=1.147695351 podStartE2EDuration="1.147695351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:50:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:50:41.146849597 +0000 UTC m=+2353.693574627" watchObservedRunningTime="2026-07-02 19:50:41.147695351 +0000 UTC m=+2353.694420357" Jul 02 19:50:41.488474 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:41.488437 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:41.493424 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:41.493400 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:42.133936 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:42.133903 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:42.134777 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:42.134757 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:43.118487 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:43.118459 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" Jul 02 19:50:49.169897 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:49.169865 2556 generic.go:356] "Generic (PLEG): container finished" podID="e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" containerID="9f4cc9ebec07ec768b7075fd666ac70cc61df8e2bee5bc302908bd87013759f1" exitCode=0 Jul 02 19:50:49.170340 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:49.169920 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" event={"ID":"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c","Type":"ContainerDied","Data":"9f4cc9ebec07ec768b7075fd666ac70cc61df8e2bee5bc302908bd87013759f1"} Jul 02 19:50:49.170340 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:49.170306 2556 scope.go:117] "RemoveContainer" containerID="9f4cc9ebec07ec768b7075fd666ac70cc61df8e2bee5bc302908bd87013759f1" Jul 02 19:50:49.171280 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:49.171249 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:49.171381 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:49.171321 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:49.171381 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:49.171352 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:50.093027 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:50.092987 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:50.176469 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:50.176432 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" event={"ID":"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c","Type":"ContainerStarted","Data":"2177bfb525fd5c9598ec8d552ada1631fb1eccae59c479b0b9ce353788c462f4"} Jul 02 19:50:50.863115 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:50.863074 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:50.864220 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:50.864188 2556 patch_prober.go:28] interesting pod/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 02 19:50:50.864371 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:50.864245 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" podUID="e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 02 19:50:51.863528 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:51.863481 2556 patch_prober.go:28] interesting pod/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 02 19:50:51.863928 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:51.863555 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" podUID="e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 02 19:50:52.607996 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:52.607308 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:50:52.863373 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:52.863283 2556 patch_prober.go:28] interesting pod/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 02 19:50:52.863373 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:52.863351 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" podUID="e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 02 19:50:53.863895 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:53.863850 2556 patch_prober.go:28] interesting pod/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 02 19:50:53.864395 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:53.863916 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" podUID="e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 02 19:50:54.202064 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:54.202019 2556 generic.go:356] "Generic (PLEG): container finished" podID="4751daa8-9b94-4270-ac05-59f561afeefb" containerID="a8cd00fc1f27400a22e535c5b8acf5c3b08aab0f07eff5f730853451d4de8994" exitCode=0 Jul 02 19:50:54.202340 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:54.202098 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" event={"ID":"4751daa8-9b94-4270-ac05-59f561afeefb","Type":"ContainerDied","Data":"a8cd00fc1f27400a22e535c5b8acf5c3b08aab0f07eff5f730853451d4de8994"} Jul 02 19:50:54.202634 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:54.202614 2556 scope.go:117] "RemoveContainer" containerID="a8cd00fc1f27400a22e535c5b8acf5c3b08aab0f07eff5f730853451d4de8994" Jul 02 19:50:54.203947 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:54.203910 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:54.204049 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:54.203997 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:54.204114 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:54.204049 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:50:54.864066 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:54.864020 2556 patch_prober.go:28] interesting pod/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 02 19:50:54.864476 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:54.864078 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" podUID="e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 02 19:50:55.212359 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:55.212316 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" event={"ID":"4751daa8-9b94-4270-ac05-59f561afeefb","Type":"ContainerStarted","Data":"8c51858b912656ffa84284f3898247586e5c6654813a945cb135473a0030fe49"} Jul 02 19:50:55.488948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:55.488858 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:55.490292 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:55.490259 2556 patch_prober.go:28] interesting pod/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 02 19:50:55.490430 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:55.490338 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" podUID="4751daa8-9b94-4270-ac05-59f561afeefb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 02 19:50:55.867282 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:55.867208 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:55.867650 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:55.867541 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:55.868253 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:55.868238 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:50:56.489079 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:56.489041 2556 patch_prober.go:28] interesting pod/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 02 19:50:56.489318 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:56.489109 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" podUID="4751daa8-9b94-4270-ac05-59f561afeefb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 02 19:50:57.134264 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:57.134230 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:57.489643 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:57.489175 2556 patch_prober.go:28] interesting pod/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 02 19:50:57.489643 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:57.489260 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" podUID="4751daa8-9b94-4270-ac05-59f561afeefb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 02 19:50:58.488644 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:58.488607 2556 patch_prober.go:28] interesting pod/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 02 19:50:58.489067 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:58.488665 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" podUID="4751daa8-9b94-4270-ac05-59f561afeefb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 02 19:50:59.492863 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:59.492827 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:50:59.493767 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:50:59.493749 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:51:05.704937 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:51:05.704895 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:51:21.802897 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:51:21.802852 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:51:35.109158 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:51:35.109098 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:51:49.705735 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:51:49.705690 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:52:06.895955 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:52:06.895866 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:52:22.408472 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:52:22.408431 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:52:37.400921 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:52:37.400884 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:52:54.716334 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:52:54.716298 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:53:11.900516 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:53:11.900477 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:53:29.708614 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:53:29.708578 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:53:51.305646 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:53:51.305609 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:54:04.111927 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:54:04.111889 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:54:22.314066 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:54:22.314024 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:54:37.301220 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:54:37.301180 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:54:53.907813 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:54:53.907733 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:55:16.311341 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:16.311305 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:55:30.312425 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:30.312387 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:55:50.207441 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:50.207406 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:55:56.586105 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:56.586068 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx"] Jul 02 19:55:56.588553 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:56.586344 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" podUID="837919fb-335a-46ed-85ea-0cb88b508dcc" containerName="maas-api" containerID="cri-o://717fa1df17fdf19f47de15465296d8cdc1a817a4024707c33b757ff700b111a6" gracePeriod=30 Jul 02 19:55:56.834785 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:56.834751 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" Jul 02 19:55:56.866033 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:56.865950 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/837919fb-335a-46ed-85ea-0cb88b508dcc-openshift-service-ca\") pod \"837919fb-335a-46ed-85ea-0cb88b508dcc\" (UID: \"837919fb-335a-46ed-85ea-0cb88b508dcc\") " Jul 02 19:55:56.866033 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:56.865998 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/837919fb-335a-46ed-85ea-0cb88b508dcc-maas-api-tls\") pod \"837919fb-335a-46ed-85ea-0cb88b508dcc\" (UID: \"837919fb-335a-46ed-85ea-0cb88b508dcc\") " Jul 02 19:55:56.866271 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:56.866057 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dp5tk\" (UniqueName: \"kubernetes.io/projected/837919fb-335a-46ed-85ea-0cb88b508dcc-kube-api-access-dp5tk\") pod \"837919fb-335a-46ed-85ea-0cb88b508dcc\" (UID: \"837919fb-335a-46ed-85ea-0cb88b508dcc\") " Jul 02 19:55:56.866391 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:56.866357 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/837919fb-335a-46ed-85ea-0cb88b508dcc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "837919fb-335a-46ed-85ea-0cb88b508dcc" (UID: "837919fb-335a-46ed-85ea-0cb88b508dcc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:55:56.868093 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:56.868071 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/837919fb-335a-46ed-85ea-0cb88b508dcc-kube-api-access-dp5tk" (OuterVolumeSpecName: "kube-api-access-dp5tk") pod "837919fb-335a-46ed-85ea-0cb88b508dcc" (UID: "837919fb-335a-46ed-85ea-0cb88b508dcc"). InnerVolumeSpecName "kube-api-access-dp5tk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:55:56.868093 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:56.868083 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/837919fb-335a-46ed-85ea-0cb88b508dcc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "837919fb-335a-46ed-85ea-0cb88b508dcc" (UID: "837919fb-335a-46ed-85ea-0cb88b508dcc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:55:56.967534 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:56.967499 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dp5tk\" (UniqueName: \"kubernetes.io/projected/837919fb-335a-46ed-85ea-0cb88b508dcc-kube-api-access-dp5tk\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:55:56.967534 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:56.967530 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/837919fb-335a-46ed-85ea-0cb88b508dcc-openshift-service-ca\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:55:56.967534 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:56.967540 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/837919fb-335a-46ed-85ea-0cb88b508dcc-maas-api-tls\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:55:57.641869 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:57.641837 2556 generic.go:356] "Generic (PLEG): container finished" podID="837919fb-335a-46ed-85ea-0cb88b508dcc" containerID="717fa1df17fdf19f47de15465296d8cdc1a817a4024707c33b757ff700b111a6" exitCode=0 Jul 02 19:55:57.642375 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:57.641880 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" Jul 02 19:55:57.642375 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:57.641926 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" event={"ID":"837919fb-335a-46ed-85ea-0cb88b508dcc","Type":"ContainerDied","Data":"717fa1df17fdf19f47de15465296d8cdc1a817a4024707c33b757ff700b111a6"} Jul 02 19:55:57.642375 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:57.641985 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx" event={"ID":"837919fb-335a-46ed-85ea-0cb88b508dcc","Type":"ContainerDied","Data":"f26314266e6b59bebb87903f22ac706dd7f1a6fe32fa3048e5f1b9a125bf5ad5"} Jul 02 19:55:57.642375 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:57.642019 2556 scope.go:117] "RemoveContainer" containerID="717fa1df17fdf19f47de15465296d8cdc1a817a4024707c33b757ff700b111a6" Jul 02 19:55:57.651565 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:57.651479 2556 scope.go:117] "RemoveContainer" containerID="717fa1df17fdf19f47de15465296d8cdc1a817a4024707c33b757ff700b111a6" Jul 02 19:55:57.651780 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:55:57.651758 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"717fa1df17fdf19f47de15465296d8cdc1a817a4024707c33b757ff700b111a6\": container with ID starting with 717fa1df17fdf19f47de15465296d8cdc1a817a4024707c33b757ff700b111a6 not found: ID does not exist" containerID="717fa1df17fdf19f47de15465296d8cdc1a817a4024707c33b757ff700b111a6" Jul 02 19:55:57.651827 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:57.651790 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"717fa1df17fdf19f47de15465296d8cdc1a817a4024707c33b757ff700b111a6"} err="failed to get container status \"717fa1df17fdf19f47de15465296d8cdc1a817a4024707c33b757ff700b111a6\": rpc error: code = NotFound desc = could not find container \"717fa1df17fdf19f47de15465296d8cdc1a817a4024707c33b757ff700b111a6\": container with ID starting with 717fa1df17fdf19f47de15465296d8cdc1a817a4024707c33b757ff700b111a6 not found: ID does not exist" Jul 02 19:55:57.663682 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:57.663654 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx"] Jul 02 19:55:57.667441 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:57.667421 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-2fca4d-856b5784cb-tjtrx"] Jul 02 19:55:57.977874 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:55:57.977842 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="837919fb-335a-46ed-85ea-0cb88b508dcc" path="/var/lib/kubelet/pods/837919fb-335a-46ed-85ea-0cb88b508dcc/volumes" Jul 02 19:56:04.306749 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:04.306711 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:56:33.453301 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:33.453265 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv"] Jul 02 19:56:33.453807 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:33.453504 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" podUID="e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" containerName="istio-proxy" containerID="cri-o://2177bfb525fd5c9598ec8d552ada1631fb1eccae59c479b0b9ce353788c462f4" gracePeriod=30 Jul 02 19:56:36.304666 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:36.304631 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c28dd"] Jul 02 19:56:38.696206 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.696181 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:56:38.716118 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.715992 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqgd5\" (UniqueName: \"kubernetes.io/projected/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-kube-api-access-xqgd5\") pod \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " Jul 02 19:56:38.716118 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.716040 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-podinfo\") pod \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " Jul 02 19:56:38.716118 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.716076 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istiod-ca-cert\") pod \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " Jul 02 19:56:38.716444 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.716133 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-workload-certs\") pod \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " Jul 02 19:56:38.716444 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.716184 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-credential-socket\") pod \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " Jul 02 19:56:38.716444 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.716206 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-data\") pod \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " Jul 02 19:56:38.716444 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.716242 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-workload-socket\") pod \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " Jul 02 19:56:38.716444 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.716275 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-envoy\") pod \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " Jul 02 19:56:38.716444 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.716302 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-token\") pod \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\" (UID: \"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c\") " Jul 02 19:56:38.716821 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.716569 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" (UID: "e7f3aeb8-3c79-4fc7-be42-37bd18ef682c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:56:38.717909 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.716943 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" (UID: "e7f3aeb8-3c79-4fc7-be42-37bd18ef682c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:56:38.717909 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.717098 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" (UID: "e7f3aeb8-3c79-4fc7-be42-37bd18ef682c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:56:38.717909 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.717769 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" (UID: "e7f3aeb8-3c79-4fc7-be42-37bd18ef682c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:56:38.717909 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.717874 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-data" (OuterVolumeSpecName: "istio-data") pod "e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" (UID: "e7f3aeb8-3c79-4fc7-be42-37bd18ef682c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:56:38.719129 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.719089 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-kube-api-access-xqgd5" (OuterVolumeSpecName: "kube-api-access-xqgd5") pod "e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" (UID: "e7f3aeb8-3c79-4fc7-be42-37bd18ef682c"). InnerVolumeSpecName "kube-api-access-xqgd5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:56:38.719823 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.719538 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" (UID: "e7f3aeb8-3c79-4fc7-be42-37bd18ef682c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:56:38.720087 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.720054 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" (UID: "e7f3aeb8-3c79-4fc7-be42-37bd18ef682c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:56:38.720412 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.720389 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-token" (OuterVolumeSpecName: "istio-token") pod "e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" (UID: "e7f3aeb8-3c79-4fc7-be42-37bd18ef682c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:56:38.817194 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.817126 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-credential-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:56:38.817194 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.817188 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-data\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:56:38.817194 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.817199 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-workload-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:56:38.817441 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.817208 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-envoy\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:56:38.817441 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.817217 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-token\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:56:38.817441 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.817226 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqgd5\" (UniqueName: \"kubernetes.io/projected/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-kube-api-access-xqgd5\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:56:38.817441 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.817234 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istio-podinfo\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:56:38.817441 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.817242 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-istiod-ca-cert\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:56:38.817441 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.817250 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c-workload-certs\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:56:38.841661 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.841625 2556 generic.go:356] "Generic (PLEG): container finished" podID="e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" containerID="2177bfb525fd5c9598ec8d552ada1631fb1eccae59c479b0b9ce353788c462f4" exitCode=0 Jul 02 19:56:38.841661 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.841661 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" event={"ID":"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c","Type":"ContainerDied","Data":"2177bfb525fd5c9598ec8d552ada1631fb1eccae59c479b0b9ce353788c462f4"} Jul 02 19:56:38.841866 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.841678 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" Jul 02 19:56:38.841866 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.841704 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv" event={"ID":"e7f3aeb8-3c79-4fc7-be42-37bd18ef682c","Type":"ContainerDied","Data":"c195ea58fb251ee5390e8fad3309a2b7b9da9c5b41e291e6f9bee4e4a2ead442"} Jul 02 19:56:38.841866 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.841728 2556 scope.go:117] "RemoveContainer" containerID="2177bfb525fd5c9598ec8d552ada1631fb1eccae59c479b0b9ce353788c462f4" Jul 02 19:56:38.856201 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.856177 2556 scope.go:117] "RemoveContainer" containerID="9f4cc9ebec07ec768b7075fd666ac70cc61df8e2bee5bc302908bd87013759f1" Jul 02 19:56:38.863992 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.863971 2556 scope.go:117] "RemoveContainer" containerID="2177bfb525fd5c9598ec8d552ada1631fb1eccae59c479b0b9ce353788c462f4" Jul 02 19:56:38.864270 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:56:38.864244 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2177bfb525fd5c9598ec8d552ada1631fb1eccae59c479b0b9ce353788c462f4\": container with ID starting with 2177bfb525fd5c9598ec8d552ada1631fb1eccae59c479b0b9ce353788c462f4 not found: ID does not exist" containerID="2177bfb525fd5c9598ec8d552ada1631fb1eccae59c479b0b9ce353788c462f4" Jul 02 19:56:38.864422 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.864274 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2177bfb525fd5c9598ec8d552ada1631fb1eccae59c479b0b9ce353788c462f4"} err="failed to get container status \"2177bfb525fd5c9598ec8d552ada1631fb1eccae59c479b0b9ce353788c462f4\": rpc error: code = NotFound desc = could not find container \"2177bfb525fd5c9598ec8d552ada1631fb1eccae59c479b0b9ce353788c462f4\": container with ID starting with 2177bfb525fd5c9598ec8d552ada1631fb1eccae59c479b0b9ce353788c462f4 not found: ID does not exist" Jul 02 19:56:38.864422 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.864295 2556 scope.go:117] "RemoveContainer" containerID="9f4cc9ebec07ec768b7075fd666ac70cc61df8e2bee5bc302908bd87013759f1" Jul 02 19:56:38.864641 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:56:38.864606 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f4cc9ebec07ec768b7075fd666ac70cc61df8e2bee5bc302908bd87013759f1\": container with ID starting with 9f4cc9ebec07ec768b7075fd666ac70cc61df8e2bee5bc302908bd87013759f1 not found: ID does not exist" containerID="9f4cc9ebec07ec768b7075fd666ac70cc61df8e2bee5bc302908bd87013759f1" Jul 02 19:56:38.864745 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.864641 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f4cc9ebec07ec768b7075fd666ac70cc61df8e2bee5bc302908bd87013759f1"} err="failed to get container status \"9f4cc9ebec07ec768b7075fd666ac70cc61df8e2bee5bc302908bd87013759f1\": rpc error: code = NotFound desc = could not find container \"9f4cc9ebec07ec768b7075fd666ac70cc61df8e2bee5bc302908bd87013759f1\": container with ID starting with 9f4cc9ebec07ec768b7075fd666ac70cc61df8e2bee5bc302908bd87013759f1 not found: ID does not exist" Jul 02 19:56:38.866947 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.866928 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv"] Jul 02 19:56:38.870764 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:38.870741 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-2fca4d-openshift-default-57d675cf9-b76nv"] Jul 02 19:56:39.978318 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:39.978284 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" path="/var/lib/kubelet/pods/e7f3aeb8-3c79-4fc7-be42-37bd18ef682c/volumes" Jul 02 19:56:40.765599 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:40.765562 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs"] Jul 02 19:56:40.765813 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:40.765788 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" podUID="4751daa8-9b94-4270-ac05-59f561afeefb" containerName="istio-proxy" containerID="cri-o://8c51858b912656ffa84284f3898247586e5c6654813a945cb135473a0030fe49" gracePeriod=30 Jul 02 19:56:44.495241 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:44.495208 2556 patch_prober.go:28] interesting pod/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 02 19:56:44.495612 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:44.495289 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" podUID="4751daa8-9b94-4270-ac05-59f561afeefb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 19:56:46.007574 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.007549 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:56:46.064870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.064790 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4751daa8-9b94-4270-ac05-59f561afeefb-istiod-ca-cert\") pod \"4751daa8-9b94-4270-ac05-59f561afeefb\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " Jul 02 19:56:46.064870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.064827 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-workload-socket\") pod \"4751daa8-9b94-4270-ac05-59f561afeefb\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " Jul 02 19:56:46.064870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.064853 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-credential-socket\") pod \"4751daa8-9b94-4270-ac05-59f561afeefb\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " Jul 02 19:56:46.065167 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.064876 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4751daa8-9b94-4270-ac05-59f561afeefb-istio-token\") pod \"4751daa8-9b94-4270-ac05-59f561afeefb\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " Jul 02 19:56:46.065167 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.064895 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvhg5\" (UniqueName: \"kubernetes.io/projected/4751daa8-9b94-4270-ac05-59f561afeefb-kube-api-access-nvhg5\") pod \"4751daa8-9b94-4270-ac05-59f561afeefb\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " Jul 02 19:56:46.065167 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.064926 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-workload-certs\") pod \"4751daa8-9b94-4270-ac05-59f561afeefb\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " Jul 02 19:56:46.065167 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.064952 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4751daa8-9b94-4270-ac05-59f561afeefb-istio-podinfo\") pod \"4751daa8-9b94-4270-ac05-59f561afeefb\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " Jul 02 19:56:46.065167 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.064988 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-istio-envoy\") pod \"4751daa8-9b94-4270-ac05-59f561afeefb\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " Jul 02 19:56:46.065167 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.065021 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-istio-data\") pod \"4751daa8-9b94-4270-ac05-59f561afeefb\" (UID: \"4751daa8-9b94-4270-ac05-59f561afeefb\") " Jul 02 19:56:46.065167 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.065136 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4751daa8-9b94-4270-ac05-59f561afeefb" (UID: "4751daa8-9b94-4270-ac05-59f561afeefb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:56:46.065893 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.065301 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4751daa8-9b94-4270-ac05-59f561afeefb" (UID: "4751daa8-9b94-4270-ac05-59f561afeefb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:56:46.065893 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.065335 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4751daa8-9b94-4270-ac05-59f561afeefb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4751daa8-9b94-4270-ac05-59f561afeefb" (UID: "4751daa8-9b94-4270-ac05-59f561afeefb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:56:46.065893 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.065319 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4751daa8-9b94-4270-ac05-59f561afeefb" (UID: "4751daa8-9b94-4270-ac05-59f561afeefb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:56:46.065893 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.065349 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-workload-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:56:46.066120 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.066042 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-istio-data" (OuterVolumeSpecName: "istio-data") pod "4751daa8-9b94-4270-ac05-59f561afeefb" (UID: "4751daa8-9b94-4270-ac05-59f561afeefb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:56:46.067285 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.067254 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4751daa8-9b94-4270-ac05-59f561afeefb-kube-api-access-nvhg5" (OuterVolumeSpecName: "kube-api-access-nvhg5") pod "4751daa8-9b94-4270-ac05-59f561afeefb" (UID: "4751daa8-9b94-4270-ac05-59f561afeefb"). InnerVolumeSpecName "kube-api-access-nvhg5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:56:46.067411 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.067263 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4751daa8-9b94-4270-ac05-59f561afeefb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4751daa8-9b94-4270-ac05-59f561afeefb" (UID: "4751daa8-9b94-4270-ac05-59f561afeefb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:56:46.067411 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.067358 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4751daa8-9b94-4270-ac05-59f561afeefb" (UID: "4751daa8-9b94-4270-ac05-59f561afeefb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:56:46.067411 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.067390 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4751daa8-9b94-4270-ac05-59f561afeefb-istio-token" (OuterVolumeSpecName: "istio-token") pod "4751daa8-9b94-4270-ac05-59f561afeefb" (UID: "4751daa8-9b94-4270-ac05-59f561afeefb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:56:46.166256 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.166218 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4751daa8-9b94-4270-ac05-59f561afeefb-istiod-ca-cert\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:56:46.166256 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.166247 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-credential-socket\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:56:46.166256 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.166256 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4751daa8-9b94-4270-ac05-59f561afeefb-istio-token\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:56:46.166483 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.166266 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvhg5\" (UniqueName: \"kubernetes.io/projected/4751daa8-9b94-4270-ac05-59f561afeefb-kube-api-access-nvhg5\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:56:46.166483 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.166276 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-workload-certs\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:56:46.166483 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.166284 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4751daa8-9b94-4270-ac05-59f561afeefb-istio-podinfo\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:56:46.166483 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.166291 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-istio-envoy\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:56:46.166483 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.166299 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4751daa8-9b94-4270-ac05-59f561afeefb-istio-data\") on node \"ip-10-0-132-104.ec2.internal\" DevicePath \"\"" Jul 02 19:56:46.823258 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.823228 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6974d4f959-5hkj9_396a53d7-c019-4512-a7ea-e7ab9ed0f53d/copy-manifests/0.log" Jul 02 19:56:46.841763 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.841727 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6974d4f959-5hkj9_396a53d7-c019-4512-a7ea-e7ab9ed0f53d/manager/0.log" Jul 02 19:56:46.879459 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.879423 2556 generic.go:356] "Generic (PLEG): container finished" podID="4751daa8-9b94-4270-ac05-59f561afeefb" containerID="8c51858b912656ffa84284f3898247586e5c6654813a945cb135473a0030fe49" exitCode=0 Jul 02 19:56:46.879649 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.879488 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" event={"ID":"4751daa8-9b94-4270-ac05-59f561afeefb","Type":"ContainerDied","Data":"8c51858b912656ffa84284f3898247586e5c6654813a945cb135473a0030fe49"} Jul 02 19:56:46.879649 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.879495 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" Jul 02 19:56:46.879649 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.879556 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs" event={"ID":"4751daa8-9b94-4270-ac05-59f561afeefb","Type":"ContainerDied","Data":"37ea281ea90b31cdfca6c29a87e074e67b829aa2d322f85ac8ccd09f9983ad71"} Jul 02 19:56:46.879649 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.879578 2556 scope.go:117] "RemoveContainer" containerID="8c51858b912656ffa84284f3898247586e5c6654813a945cb135473a0030fe49" Jul 02 19:56:46.888852 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.888698 2556 scope.go:117] "RemoveContainer" containerID="a8cd00fc1f27400a22e535c5b8acf5c3b08aab0f07eff5f730853451d4de8994" Jul 02 19:56:46.896365 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.896344 2556 scope.go:117] "RemoveContainer" containerID="8c51858b912656ffa84284f3898247586e5c6654813a945cb135473a0030fe49" Jul 02 19:56:46.896608 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:56:46.896589 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c51858b912656ffa84284f3898247586e5c6654813a945cb135473a0030fe49\": container with ID starting with 8c51858b912656ffa84284f3898247586e5c6654813a945cb135473a0030fe49 not found: ID does not exist" containerID="8c51858b912656ffa84284f3898247586e5c6654813a945cb135473a0030fe49" Jul 02 19:56:46.896673 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.896621 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c51858b912656ffa84284f3898247586e5c6654813a945cb135473a0030fe49"} err="failed to get container status \"8c51858b912656ffa84284f3898247586e5c6654813a945cb135473a0030fe49\": rpc error: code = NotFound desc = could not find container \"8c51858b912656ffa84284f3898247586e5c6654813a945cb135473a0030fe49\": container with ID starting with 8c51858b912656ffa84284f3898247586e5c6654813a945cb135473a0030fe49 not found: ID does not exist" Jul 02 19:56:46.896673 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.896648 2556 scope.go:117] "RemoveContainer" containerID="a8cd00fc1f27400a22e535c5b8acf5c3b08aab0f07eff5f730853451d4de8994" Jul 02 19:56:46.896854 ip-10-0-132-104 kubenswrapper[2556]: E0702 19:56:46.896838 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8cd00fc1f27400a22e535c5b8acf5c3b08aab0f07eff5f730853451d4de8994\": container with ID starting with a8cd00fc1f27400a22e535c5b8acf5c3b08aab0f07eff5f730853451d4de8994 not found: ID does not exist" containerID="a8cd00fc1f27400a22e535c5b8acf5c3b08aab0f07eff5f730853451d4de8994" Jul 02 19:56:46.896910 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.896865 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8cd00fc1f27400a22e535c5b8acf5c3b08aab0f07eff5f730853451d4de8994"} err="failed to get container status \"a8cd00fc1f27400a22e535c5b8acf5c3b08aab0f07eff5f730853451d4de8994\": rpc error: code = NotFound desc = could not find container \"a8cd00fc1f27400a22e535c5b8acf5c3b08aab0f07eff5f730853451d4de8994\": container with ID starting with a8cd00fc1f27400a22e535c5b8acf5c3b08aab0f07eff5f730853451d4de8994 not found: ID does not exist" Jul 02 19:56:46.901948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.901927 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs"] Jul 02 19:56:46.905728 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:46.905707 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-f9d728-openshift-default-58d949b847-sqmcs"] Jul 02 19:56:47.954000 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:47.953972 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq_1c48642e-167b-466b-91a4-08490e74cf80/extract/0.log" Jul 02 19:56:47.960416 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:47.960398 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq_1c48642e-167b-466b-91a4-08490e74cf80/util/0.log" Jul 02 19:56:47.967114 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:47.967084 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq_1c48642e-167b-466b-91a4-08490e74cf80/pull/0.log" Jul 02 19:56:47.978092 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:47.978071 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4751daa8-9b94-4270-ac05-59f561afeefb" path="/var/lib/kubelet/pods/4751daa8-9b94-4270-ac05-59f561afeefb/volumes" Jul 02 19:56:48.076168 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:48.076135 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65_edfcfd82-124a-4ff2-be30-a65941f85ce5/util/0.log" Jul 02 19:56:48.082765 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:48.082739 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65_edfcfd82-124a-4ff2-be30-a65941f85ce5/pull/0.log" Jul 02 19:56:48.089451 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:48.089411 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65_edfcfd82-124a-4ff2-be30-a65941f85ce5/extract/0.log" Jul 02 19:56:48.199541 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:48.199503 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5_ed6128c8-16a2-4a8a-91a9-faa49567dfcb/pull/0.log" Jul 02 19:56:48.205948 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:48.205881 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5_ed6128c8-16a2-4a8a-91a9-faa49567dfcb/extract/0.log" Jul 02 19:56:48.212244 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:48.212225 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5_ed6128c8-16a2-4a8a-91a9-faa49567dfcb/util/0.log" Jul 02 19:56:48.321740 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:48.321708 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7_5daaf506-24ce-43da-9389-c59d0d44f1bb/pull/0.log" Jul 02 19:56:48.327894 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:48.327868 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7_5daaf506-24ce-43da-9389-c59d0d44f1bb/extract/0.log" Jul 02 19:56:48.333536 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:48.333517 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7_5daaf506-24ce-43da-9389-c59d0d44f1bb/util/0.log" Jul 02 19:56:49.145510 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:49.145482 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-c28dd_ad960ad7-f120-4c3c-a8fd-330fbf306105/limitador/0.log" Jul 02 19:56:49.741849 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:49.741813 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vhsn8_58bcf208-b1c1-4b43-ba8d-1877473c0582/discovery/0.log" Jul 02 19:56:51.092987 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:51.092938 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbszl2j_3df767f1-f99c-4e57-a70f-294ad26b3f1a/main/0.log" Jul 02 19:56:58.567896 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:58.567863 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bhgfb_882fce17-202b-44a2-9002-efdcb888f969/global-pull-secret-syncer/0.log" Jul 02 19:56:58.703031 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:58.702999 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-c5wmt_4a8d6b85-07f6-4d4c-86d4-3dadb68f48af/konnectivity-agent/0.log" Jul 02 19:56:58.756142 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:56:58.756104 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-104.ec2.internal_d9ea7e94a50077843ac883931f2f632a/haproxy/0.log" Jul 02 19:57:02.315412 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:02.315377 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq_1c48642e-167b-466b-91a4-08490e74cf80/extract/0.log" Jul 02 19:57:02.336692 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:02.336663 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq_1c48642e-167b-466b-91a4-08490e74cf80/util/0.log" Jul 02 19:57:02.359477 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:02.359444 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvntq_1c48642e-167b-466b-91a4-08490e74cf80/pull/0.log" Jul 02 19:57:02.393788 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:02.393755 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65_edfcfd82-124a-4ff2-be30-a65941f85ce5/extract/0.log" Jul 02 19:57:02.416439 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:02.416407 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65_edfcfd82-124a-4ff2-be30-a65941f85ce5/util/0.log" Jul 02 19:57:02.439060 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:02.439029 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04md65_edfcfd82-124a-4ff2-be30-a65941f85ce5/pull/0.log" Jul 02 19:57:02.465904 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:02.465877 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5_ed6128c8-16a2-4a8a-91a9-faa49567dfcb/extract/0.log" Jul 02 19:57:02.488023 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:02.487993 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5_ed6128c8-16a2-4a8a-91a9-faa49567dfcb/util/0.log" Jul 02 19:57:02.509609 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:02.509577 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735dhk5_ed6128c8-16a2-4a8a-91a9-faa49567dfcb/pull/0.log" Jul 02 19:57:02.535202 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:02.535169 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7_5daaf506-24ce-43da-9389-c59d0d44f1bb/extract/0.log" Jul 02 19:57:02.559240 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:02.559216 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7_5daaf506-24ce-43da-9389-c59d0d44f1bb/util/0.log" Jul 02 19:57:02.598926 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:02.598833 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1shpg7_5daaf506-24ce-43da-9389-c59d0d44f1bb/pull/0.log" Jul 02 19:57:03.411905 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:03.411877 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-c28dd_ad960ad7-f120-4c3c-a8fd-330fbf306105/limitador/0.log" Jul 02 19:57:05.719711 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:05.719679 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ls6fs_78caeef4-8166-4bba-b486-d1ba7cd02385/node-exporter/0.log" Jul 02 19:57:05.738724 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:05.738699 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ls6fs_78caeef4-8166-4bba-b486-d1ba7cd02385/kube-rbac-proxy/0.log" Jul 02 19:57:05.764208 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:05.764179 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ls6fs_78caeef4-8166-4bba-b486-d1ba7cd02385/init-textfile/0.log" Jul 02 19:57:07.807889 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:07.807866 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-x6qcl_535b962d-6d94-4e02-bd83-4411a9f5fd05/networking-console-plugin/0.log" Jul 02 19:57:09.043294 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.043254 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz"] Jul 02 19:57:09.043764 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.043632 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" containerName="istio-proxy" Jul 02 19:57:09.043764 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.043649 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" containerName="istio-proxy" Jul 02 19:57:09.043764 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.043660 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="290ec0dc-cfc7-46c8-a865-17f68f47701e" containerName="istio-proxy" Jul 02 19:57:09.043764 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.043668 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="290ec0dc-cfc7-46c8-a865-17f68f47701e" containerName="istio-proxy" Jul 02 19:57:09.043764 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.043680 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4751daa8-9b94-4270-ac05-59f561afeefb" containerName="istio-proxy" Jul 02 19:57:09.043764 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.043688 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4751daa8-9b94-4270-ac05-59f561afeefb" containerName="istio-proxy" Jul 02 19:57:09.043764 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.043706 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="837919fb-335a-46ed-85ea-0cb88b508dcc" containerName="maas-api" Jul 02 19:57:09.043764 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.043713 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="837919fb-335a-46ed-85ea-0cb88b508dcc" containerName="maas-api" Jul 02 19:57:09.043764 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.043725 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4751daa8-9b94-4270-ac05-59f561afeefb" containerName="istio-proxy" Jul 02 19:57:09.043764 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.043733 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4751daa8-9b94-4270-ac05-59f561afeefb" containerName="istio-proxy" Jul 02 19:57:09.044301 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.043805 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7f3aeb8-3c79-4fc7-be42-37bd18ef682c" containerName="istio-proxy" Jul 02 19:57:09.044301 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.043817 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="837919fb-335a-46ed-85ea-0cb88b508dcc" containerName="maas-api" Jul 02 19:57:09.044301 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.043827 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4751daa8-9b94-4270-ac05-59f561afeefb" containerName="istio-proxy" Jul 02 19:57:09.044301 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.043836 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="290ec0dc-cfc7-46c8-a865-17f68f47701e" containerName="istio-proxy" Jul 02 19:57:09.046892 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.046873 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" Jul 02 19:57:09.049595 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.049575 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xzntd\"/\"openshift-service-ca.crt\"" Jul 02 19:57:09.050672 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.050648 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xzntd\"/\"kube-root-ca.crt\"" Jul 02 19:57:09.050827 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.050662 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xzntd\"/\"default-dockercfg-d7pj2\"" Jul 02 19:57:09.056876 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.056848 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz"] Jul 02 19:57:09.149421 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.149380 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/14a623db-b90a-42da-9f64-bc9571f96beb-lib-modules\") pod \"perf-node-gather-daemonset-ckgcz\" (UID: \"14a623db-b90a-42da-9f64-bc9571f96beb\") " pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" Jul 02 19:57:09.149605 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.149443 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/14a623db-b90a-42da-9f64-bc9571f96beb-podres\") pod \"perf-node-gather-daemonset-ckgcz\" (UID: \"14a623db-b90a-42da-9f64-bc9571f96beb\") " pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" Jul 02 19:57:09.149605 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.149520 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/14a623db-b90a-42da-9f64-bc9571f96beb-proc\") pod \"perf-node-gather-daemonset-ckgcz\" (UID: \"14a623db-b90a-42da-9f64-bc9571f96beb\") " pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" Jul 02 19:57:09.149605 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.149551 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qts5\" (UniqueName: \"kubernetes.io/projected/14a623db-b90a-42da-9f64-bc9571f96beb-kube-api-access-8qts5\") pod \"perf-node-gather-daemonset-ckgcz\" (UID: \"14a623db-b90a-42da-9f64-bc9571f96beb\") " pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" Jul 02 19:57:09.149605 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.149577 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14a623db-b90a-42da-9f64-bc9571f96beb-sys\") pod \"perf-node-gather-daemonset-ckgcz\" (UID: \"14a623db-b90a-42da-9f64-bc9571f96beb\") " pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" Jul 02 19:57:09.250278 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.250241 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/14a623db-b90a-42da-9f64-bc9571f96beb-proc\") pod \"perf-node-gather-daemonset-ckgcz\" (UID: \"14a623db-b90a-42da-9f64-bc9571f96beb\") " pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" Jul 02 19:57:09.250278 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.250279 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qts5\" (UniqueName: \"kubernetes.io/projected/14a623db-b90a-42da-9f64-bc9571f96beb-kube-api-access-8qts5\") pod \"perf-node-gather-daemonset-ckgcz\" (UID: \"14a623db-b90a-42da-9f64-bc9571f96beb\") " pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" Jul 02 19:57:09.250534 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.250302 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14a623db-b90a-42da-9f64-bc9571f96beb-sys\") pod \"perf-node-gather-daemonset-ckgcz\" (UID: \"14a623db-b90a-42da-9f64-bc9571f96beb\") " pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" Jul 02 19:57:09.250534 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.250328 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/14a623db-b90a-42da-9f64-bc9571f96beb-lib-modules\") pod \"perf-node-gather-daemonset-ckgcz\" (UID: \"14a623db-b90a-42da-9f64-bc9571f96beb\") " pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" Jul 02 19:57:09.250534 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.250361 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/14a623db-b90a-42da-9f64-bc9571f96beb-podres\") pod \"perf-node-gather-daemonset-ckgcz\" (UID: \"14a623db-b90a-42da-9f64-bc9571f96beb\") " pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" Jul 02 19:57:09.250534 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.250368 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/14a623db-b90a-42da-9f64-bc9571f96beb-proc\") pod \"perf-node-gather-daemonset-ckgcz\" (UID: \"14a623db-b90a-42da-9f64-bc9571f96beb\") " pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" Jul 02 19:57:09.250534 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.250390 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14a623db-b90a-42da-9f64-bc9571f96beb-sys\") pod \"perf-node-gather-daemonset-ckgcz\" (UID: \"14a623db-b90a-42da-9f64-bc9571f96beb\") " pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" Jul 02 19:57:09.250534 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.250504 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/14a623db-b90a-42da-9f64-bc9571f96beb-lib-modules\") pod \"perf-node-gather-daemonset-ckgcz\" (UID: \"14a623db-b90a-42da-9f64-bc9571f96beb\") " pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" Jul 02 19:57:09.250534 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.250527 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/14a623db-b90a-42da-9f64-bc9571f96beb-podres\") pod \"perf-node-gather-daemonset-ckgcz\" (UID: \"14a623db-b90a-42da-9f64-bc9571f96beb\") " pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" Jul 02 19:57:09.257859 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.257835 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qts5\" (UniqueName: \"kubernetes.io/projected/14a623db-b90a-42da-9f64-bc9571f96beb-kube-api-access-8qts5\") pod \"perf-node-gather-daemonset-ckgcz\" (UID: \"14a623db-b90a-42da-9f64-bc9571f96beb\") " pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" Jul 02 19:57:09.358367 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.358270 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" Jul 02 19:57:09.485399 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.485362 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz"] Jul 02 19:57:09.487798 ip-10-0-132-104 kubenswrapper[2556]: W0702 19:57:09.487775 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod14a623db_b90a_42da_9f64_bc9571f96beb.slice/crio-a9943bdf1e80e30a8e72993714b63198d50852152a6eba35d788e01109aa6c84 WatchSource:0}: Error finding container a9943bdf1e80e30a8e72993714b63198d50852152a6eba35d788e01109aa6c84: Status 404 returned error can't find the container with id a9943bdf1e80e30a8e72993714b63198d50852152a6eba35d788e01109aa6c84 Jul 02 19:57:09.990914 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.990868 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" event={"ID":"14a623db-b90a-42da-9f64-bc9571f96beb","Type":"ContainerStarted","Data":"a54f666c037ce25e5c286af336ee873f0809ff4bf0e7521036be0b7a317c6611"} Jul 02 19:57:09.991126 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.990928 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" event={"ID":"14a623db-b90a-42da-9f64-bc9571f96beb","Type":"ContainerStarted","Data":"a9943bdf1e80e30a8e72993714b63198d50852152a6eba35d788e01109aa6c84"} Jul 02 19:57:09.991126 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:09.990959 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" Jul 02 19:57:10.012873 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:10.012812 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" podStartSLOduration=1.012791803 podStartE2EDuration="1.012791803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:57:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:57:10.005969771 +0000 UTC m=+2742.552694800" watchObservedRunningTime="2026-07-02 19:57:10.012791803 +0000 UTC m=+2742.559516810" Jul 02 19:57:10.803605 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:10.803579 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7qq96_77f8b592-4af7-48fe-9385-0de92afaf07c/dns/0.log" Jul 02 19:57:10.823983 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:10.823958 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7qq96_77f8b592-4af7-48fe-9385-0de92afaf07c/kube-rbac-proxy/0.log" Jul 02 19:57:10.991482 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:10.991458 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sv2qm_dee24e3b-8c4a-48d3-9c21-31c9ca499d72/dns-node-resolver/0.log" Jul 02 19:57:11.580360 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:11.580326 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p8crb_205e2e72-bfab-43b6-b8d3-8af9a1c8561a/node-ca/0.log" Jul 02 19:57:12.686070 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:12.686037 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vhsn8_58bcf208-b1c1-4b43-ba8d-1877473c0582/discovery/0.log" Jul 02 19:57:13.647870 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:13.647833 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8t8d8_171c244d-2dda-414a-a89f-c56e1a47c97c/serve-healthcheck-canary/0.log" Jul 02 19:57:14.250268 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:14.250239 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g99mz_608ae54a-8df6-4349-8d64-4fa26c784657/kube-rbac-proxy/0.log" Jul 02 19:57:14.270285 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:14.270259 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g99mz_608ae54a-8df6-4349-8d64-4fa26c784657/exporter/0.log" Jul 02 19:57:14.303553 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:14.303525 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g99mz_608ae54a-8df6-4349-8d64-4fa26c784657/extractor/0.log" Jul 02 19:57:16.004258 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:16.004230 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xzntd/perf-node-gather-daemonset-ckgcz" Jul 02 19:57:17.052530 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:17.052497 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6974d4f959-5hkj9_396a53d7-c019-4512-a7ea-e7ab9ed0f53d/manager/0.log" Jul 02 19:57:17.070095 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:17.070069 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6974d4f959-5hkj9_396a53d7-c019-4512-a7ea-e7ab9ed0f53d/copy-manifests/0.log" Jul 02 19:57:18.659690 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:18.659652 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5d5f586fb-gt8gk_de1ba6eb-2ad0-431d-a1e1-3211cb86cee4/manager/0.log" Jul 02 19:57:19.568279 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:19.568186 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vhsn8_58bcf208-b1c1-4b43-ba8d-1877473c0582/discovery/0.log" Jul 02 19:57:21.106021 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:21.105980 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vhsn8_58bcf208-b1c1-4b43-ba8d-1877473c0582/discovery/0.log" Jul 02 19:57:25.211799 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:25.211771 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7skzd_254f17a4-cbdb-420e-9fa7-d2a49ac7c026/kube-multus-additional-cni-plugins/0.log" Jul 02 19:57:25.235043 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:25.235011 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7skzd_254f17a4-cbdb-420e-9fa7-d2a49ac7c026/egress-router-binary-copy/0.log" Jul 02 19:57:25.255470 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:25.255439 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7skzd_254f17a4-cbdb-420e-9fa7-d2a49ac7c026/cni-plugins/0.log" Jul 02 19:57:25.276980 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:25.276951 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7skzd_254f17a4-cbdb-420e-9fa7-d2a49ac7c026/bond-cni-plugin/0.log" Jul 02 19:57:25.296797 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:25.296771 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7skzd_254f17a4-cbdb-420e-9fa7-d2a49ac7c026/routeoverride-cni/0.log" Jul 02 19:57:25.317584 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:25.317558 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7skzd_254f17a4-cbdb-420e-9fa7-d2a49ac7c026/whereabouts-cni-bincopy/0.log" Jul 02 19:57:25.337965 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:25.337940 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7skzd_254f17a4-cbdb-420e-9fa7-d2a49ac7c026/whereabouts-cni/0.log" Jul 02 19:57:25.718074 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:25.718040 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fqqsq_5d0af660-54fe-44aa-b75e-a0116a2f836d/kube-multus/0.log" Jul 02 19:57:25.857328 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:25.857296 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pjsd8_75464ead-f0fb-4ea8-beba-55b639087860/network-metrics-daemon/0.log" Jul 02 19:57:25.874548 ip-10-0-132-104 kubenswrapper[2556]: I0702 19:57:25.874523 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pjsd8_75464ead-f0fb-4ea8-beba-55b639087860/kube-rbac-proxy/0.log"