Jul 02 23:05:07.711919 ip-10-0-136-243 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 02 23:05:07.711931 ip-10-0-136-243 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 02 23:05:07.711939 ip-10-0-136-243 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 02 23:05:07.712167 ip-10-0-136-243 systemd[1]: Failed to start Kubernetes Kubelet. Jul 02 23:05:17.790550 ip-10-0-136-243 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 02 23:05:17.790571 ip-10-0-136-243 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 04899697fdf149fc8af352f828966804 -- Jul 02 23:07:15.849204 ip-10-0-136-243 systemd[1]: Starting Kubernetes Kubelet... Jul 02 23:07:16.261705 ip-10-0-136-243 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 23:07:16.261705 ip-10-0-136-243 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 23:07:16.261705 ip-10-0-136-243 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 23:07:16.261705 ip-10-0-136-243 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 23:07:16.261705 ip-10-0-136-243 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 23:07:16.263701 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.263613 2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 23:07:16.268048 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268032 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:07:16.268048 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268048 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:07:16.268116 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268051 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:07:16.268116 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268055 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:07:16.268116 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268058 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:07:16.268116 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268061 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:07:16.268116 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268064 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:07:16.268116 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268068 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:07:16.268116 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268071 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:07:16.268116 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268074 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:07:16.268116 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268076 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:07:16.268116 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268079 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:07:16.268116 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268082 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:07:16.268116 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268084 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:07:16.268116 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268087 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:07:16.268116 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268090 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:07:16.268116 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268093 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:07:16.268607 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268598 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:07:16.268607 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268607 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:07:16.268663 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268610 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:07:16.268663 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268613 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:07:16.268663 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268621 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:07:16.268663 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268623 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:07:16.268663 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268626 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:07:16.268663 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268629 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:07:16.268663 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268632 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:07:16.268663 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268634 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:07:16.268663 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268637 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:07:16.268663 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268639 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:07:16.268663 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268641 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:07:16.268663 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268644 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:07:16.268663 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268646 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:07:16.268663 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268649 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:07:16.268663 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268653 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:07:16.268663 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268657 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:07:16.268663 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268660 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:07:16.268663 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268663 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:07:16.268663 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268667 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:07:16.269137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268670 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:07:16.269137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268673 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:07:16.269137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268676 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:07:16.269137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268679 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:07:16.269137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268683 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:07:16.269137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268686 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:07:16.269137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268689 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:07:16.269137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268692 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:07:16.269137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268694 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:07:16.269137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268697 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:07:16.269137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268699 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:07:16.269137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268702 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:07:16.269137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268705 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:07:16.269137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268707 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:07:16.269137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268710 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:07:16.269137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268712 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:07:16.269137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268715 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:07:16.269137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268718 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:07:16.269137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268720 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268723 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268725 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268727 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268730 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268732 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268735 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268737 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268739 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268742 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268744 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268749 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268752 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268755 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268758 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268761 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268764 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268767 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268770 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268772 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:07:16.269602 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268775 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268778 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268780 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268783 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268785 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268788 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268790 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268793 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268796 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268798 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268801 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268810 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268813 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268816 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268821 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268823 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268826 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268828 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268830 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268833 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:07:16.270105 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268835 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268838 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.268840 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269599 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269606 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269609 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269612 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269615 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269618 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269621 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269623 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269626 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269628 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269631 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269633 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269636 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269638 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269641 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269643 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269646 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:07:16.270587 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269649 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269651 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269654 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269656 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269659 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269661 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269664 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269666 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269669 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269671 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269673 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269676 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269678 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269681 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269683 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269686 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269688 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269692 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269694 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269697 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:07:16.271191 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269699 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269702 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269704 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269707 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269709 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269712 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269714 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269717 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269719 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269722 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269724 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269726 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269729 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269731 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269734 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269736 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269738 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269741 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269743 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269748 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:07:16.271688 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269752 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:07:16.272201 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269755 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:07:16.272201 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269758 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:07:16.272201 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269760 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:07:16.272201 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269763 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:07:16.272201 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269765 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:07:16.272201 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269768 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:07:16.272201 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269770 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:07:16.272201 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269774 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:07:16.272201 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269778 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:07:16.272201 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269781 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:07:16.272201 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269784 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:07:16.272201 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269787 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:07:16.272201 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269789 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:07:16.272201 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269792 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:07:16.272201 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269795 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:07:16.272201 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269798 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:07:16.272201 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269801 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:07:16.272201 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269803 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:07:16.272201 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269806 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:07:16.272670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269808 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:07:16.272670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269811 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:07:16.272670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269813 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:07:16.272670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269817 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:07:16.272670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269819 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:07:16.272670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269821 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:07:16.272670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269824 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:07:16.272670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269826 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:07:16.272670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269829 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:07:16.272670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269831 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:07:16.272670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269834 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:07:16.272670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269836 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:07:16.272670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269839 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:07:16.272670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269841 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:07:16.272670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269844 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:07:16.272670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269846 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:07:16.272670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269849 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:07:16.272670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269851 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:07:16.272670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269854 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:07:16.273168 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269858 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:07:16.273168 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269861 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:07:16.273168 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.269864 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:07:16.273168 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270265 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:07:16.273168 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270271 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:07:16.273168 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270274 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:07:16.273168 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270276 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:07:16.273168 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270279 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:07:16.273168 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270281 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:07:16.273168 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270284 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:07:16.273168 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270288 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:07:16.273168 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270291 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:07:16.273168 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270293 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:07:16.273168 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270296 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:07:16.273168 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270299 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:07:16.273168 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270301 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:07:16.273168 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270304 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:07:16.273168 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270306 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:07:16.273168 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270309 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270311 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270314 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270316 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270319 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270322 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270324 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270327 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270329 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270332 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270334 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270337 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270340 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270342 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270345 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270347 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270349 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270352 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270354 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270357 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:07:16.273641 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270359 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:07:16.274137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270361 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:07:16.274137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270364 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:07:16.274137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270366 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:07:16.274137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270369 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:07:16.274137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270371 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:07:16.274137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270374 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:07:16.274137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270376 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:07:16.274137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270379 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:07:16.274137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270381 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:07:16.274137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270383 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:07:16.274137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270386 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:07:16.274137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270388 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:07:16.274137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270391 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:07:16.274137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270394 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:07:16.274137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270397 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:07:16.274137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270400 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:07:16.274137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270403 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:07:16.274137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270407 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:07:16.274137 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270410 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270413 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270416 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270419 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270424 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270427 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270430 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270432 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270435 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270438 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270440 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270443 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270446 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270448 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270451 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270453 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270455 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270458 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270461 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270463 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:07:16.274632 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270466 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270469 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270472 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270474 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270476 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270479 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270481 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270484 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270486 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270489 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270491 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270493 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270497 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270500 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270502 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270504 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270508 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270511 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270513 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270516 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:07:16.275130 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270518 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270521 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270525 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270528 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.270531 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270602 2583 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270608 2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270614 2583 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270619 2583 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270623 2583 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270626 2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270630 2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270634 2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270637 2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270640 2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270643 2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270646 2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270649 2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270652 2583 flags.go:64] FLAG: --cgroup-root="" Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270655 2583 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270658 2583 flags.go:64] FLAG: --client-ca-file="" Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270660 2583 flags.go:64] FLAG: --cloud-provider="external" Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270663 2583 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 23:07:16.275613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270667 2583 flags.go:64] FLAG: --cluster-domain="" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270670 2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270674 2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270677 2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270680 2583 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270685 2583 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270689 2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270692 2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270695 2583 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270698 2583 flags.go:64] FLAG: --contention-profiling="false" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270701 2583 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270704 2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270707 2583 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270709 2583 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270714 2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270717 2583 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270720 2583 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270722 2583 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270727 2583 flags.go:64] FLAG: --enable-server="true" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270729 2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270733 2583 flags.go:64] FLAG: --event-burst="100" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270736 2583 flags.go:64] FLAG: --event-qps="50" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270739 2583 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270742 2583 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 23:07:16.276229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270744 2583 flags.go:64] FLAG: --eviction-hard="" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270748 2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270751 2583 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270754 2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270757 2583 flags.go:64] FLAG: --eviction-soft="" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270760 2583 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270762 2583 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270766 2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270768 2583 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270771 2583 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270773 2583 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270777 2583 flags.go:64] FLAG: --feature-gates="" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270781 2583 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270784 2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270787 2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270790 2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270793 2583 flags.go:64] FLAG: --healthz-port="10248" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270797 2583 flags.go:64] FLAG: --help="false" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270799 2583 flags.go:64] FLAG: --hostname-override="ip-10-0-136-243.ec2.internal" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270802 2583 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270806 2583 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270808 2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270813 2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 23:07:16.276805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270816 2583 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270818 2583 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270821 2583 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270825 2583 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270828 2583 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270831 2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270834 2583 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270836 2583 flags.go:64] FLAG: --kube-reserved="" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270839 2583 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270842 2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270845 2583 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270848 2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270851 2583 flags.go:64] FLAG: --lock-file="" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270853 2583 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270856 2583 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270859 2583 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270864 2583 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270866 2583 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270869 2583 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270872 2583 flags.go:64] FLAG: --logging-format="text" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270874 2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270878 2583 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270881 2583 flags.go:64] FLAG: --manifest-url="" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270899 2583 flags.go:64] FLAG: --manifest-url-header="" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270904 2583 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 23:07:16.277402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270907 2583 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270911 2583 flags.go:64] FLAG: --max-pods="110" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270914 2583 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270917 2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270920 2583 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270923 2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270926 2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270929 2583 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270931 2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270938 2583 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270942 2583 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270945 2583 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270948 2583 flags.go:64] FLAG: --pod-cidr="" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270951 2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270956 2583 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270958 2583 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270961 2583 flags.go:64] FLAG: --pods-per-core="0" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270964 2583 flags.go:64] FLAG: --port="10250" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270967 2583 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270970 2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04788765f495f940d" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270973 2583 flags.go:64] FLAG: --qos-reserved="" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270975 2583 flags.go:64] FLAG: --read-only-port="10255" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270978 2583 flags.go:64] FLAG: --register-node="true" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270981 2583 flags.go:64] FLAG: --register-with-taints="" Jul 02 23:07:16.278036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270989 2583 flags.go:64] FLAG: --registry-burst="10" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270992 2583 flags.go:64] FLAG: --registry-qps="5" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270995 2583 flags.go:64] FLAG: --reserved-cpus="" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.270998 2583 flags.go:64] FLAG: --reserved-memory="" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271001 2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271005 2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271008 2583 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271011 2583 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271013 2583 flags.go:64] FLAG: --runonce="false" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271016 2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271019 2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271022 2583 flags.go:64] FLAG: --seccomp-default="false" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271024 2583 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271027 2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271030 2583 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271033 2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271036 2583 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271038 2583 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271042 2583 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271045 2583 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271048 2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271051 2583 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271054 2583 flags.go:64] FLAG: --system-cgroups="" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271056 2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271062 2583 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 23:07:16.278604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271065 2583 flags.go:64] FLAG: --tls-cert-file="" Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271067 2583 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271071 2583 flags.go:64] FLAG: --tls-min-version="" Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271074 2583 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271076 2583 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271079 2583 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271082 2583 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271085 2583 flags.go:64] FLAG: --v="2" Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271089 2583 flags.go:64] FLAG: --version="false" Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271093 2583 flags.go:64] FLAG: --vmodule="" Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271099 2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.271102 2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271219 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271224 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271227 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271230 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271233 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271236 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271238 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271241 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271244 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271246 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:07:16.279561 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271248 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:07:16.280441 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271251 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:07:16.280441 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271253 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:07:16.280441 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271257 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:07:16.280441 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271261 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:07:16.280441 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271263 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:07:16.280441 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271266 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:07:16.280441 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271270 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:07:16.280441 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271273 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:07:16.280441 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271276 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:07:16.280441 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271279 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:07:16.280441 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271282 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:07:16.280441 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271284 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:07:16.280441 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271287 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:07:16.280441 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271291 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:07:16.280441 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271293 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:07:16.280441 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271296 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:07:16.280441 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271298 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:07:16.280441 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271300 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:07:16.280441 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271303 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271307 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271310 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271312 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271314 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271317 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271320 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271322 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271325 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271327 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271329 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271332 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271334 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271337 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271339 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271342 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271346 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271348 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271351 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271354 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:07:16.280942 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271358 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271361 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271364 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271366 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271369 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271372 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271374 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271377 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271379 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271381 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271384 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271386 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271389 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271392 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271395 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271397 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271400 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271402 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271405 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271407 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:07:16.281427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271410 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:07:16.281952 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271412 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:07:16.281952 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271415 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:07:16.281952 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271417 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:07:16.281952 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271419 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:07:16.281952 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271422 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:07:16.281952 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271424 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:07:16.281952 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271426 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:07:16.281952 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271430 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:07:16.281952 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271432 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:07:16.281952 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271435 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:07:16.281952 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271437 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:07:16.281952 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271439 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:07:16.281952 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271442 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:07:16.281952 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271444 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:07:16.281952 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271447 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:07:16.281952 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271449 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:07:16.281952 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271452 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:07:16.281952 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271454 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:07:16.281952 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271456 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:07:16.282427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271460 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:07:16.282427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271463 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:07:16.282427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271466 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:07:16.282427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271469 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:07:16.282427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271471 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:07:16.282427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271475 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:07:16.282427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271477 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:07:16.282427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271480 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:07:16.282427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271482 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:07:16.282427 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.271484 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:07:16.282427 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.272082 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 23:07:16.282427 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.280580 2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 23:07:16.282427 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.280595 2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 23:07:16.282427 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.281274 2583 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.281286 2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281345 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281351 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281354 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281357 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281360 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281363 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281366 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281369 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281372 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281375 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281378 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281380 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281383 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281389 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281392 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281395 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281397 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281400 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:07:16.282785 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281402 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281405 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281408 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281410 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281413 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281415 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281417 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281420 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281422 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281424 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281427 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281430 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281433 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281436 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281439 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281442 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281445 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281447 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281450 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281453 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:07:16.283388 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281455 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:07:16.283869 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281457 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:07:16.283869 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281460 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:07:16.283869 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281463 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:07:16.283869 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281467 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:07:16.283869 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281471 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:07:16.283869 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281474 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:07:16.283869 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281477 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:07:16.283869 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281482 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:07:16.283869 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281486 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:07:16.283869 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281489 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:07:16.283869 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281492 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:07:16.283869 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281495 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:07:16.283869 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281498 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:07:16.283869 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281500 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:07:16.283869 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281503 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:07:16.283869 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281506 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:07:16.283869 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281509 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:07:16.283869 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281511 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281514 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281516 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281518 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281521 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281524 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281527 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281530 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281533 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281535 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281538 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281540 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281542 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281545 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281547 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281550 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281553 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281555 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281558 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281560 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:07:16.284328 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281563 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281566 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281568 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281572 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281574 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281577 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281579 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281582 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281584 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281587 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281589 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281592 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281594 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281597 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281600 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281603 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281605 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281608 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281610 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281613 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:07:16.284818 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281615 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:07:16.285312 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281617 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:07:16.285312 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281620 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:07:16.285312 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.281625 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 23:07:16.285312 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281740 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:07:16.285312 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281746 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:07:16.285312 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281749 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:07:16.285312 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281752 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:07:16.285312 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281754 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:07:16.285312 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281757 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:07:16.285312 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281760 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:07:16.285312 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281762 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:07:16.285312 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281765 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:07:16.285312 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281768 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:07:16.285312 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281770 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:07:16.285312 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281773 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:07:16.285670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281776 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:07:16.285670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281778 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:07:16.285670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281781 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:07:16.285670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281783 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:07:16.285670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281786 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:07:16.285670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281788 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:07:16.285670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281792 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:07:16.285670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281796 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:07:16.285670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281799 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:07:16.285670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281802 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:07:16.285670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281804 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:07:16.285670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281806 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:07:16.285670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281809 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:07:16.285670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281812 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:07:16.285670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281814 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:07:16.285670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281817 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:07:16.285670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281819 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:07:16.285670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281821 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:07:16.285670 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281824 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281826 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281829 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281831 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281834 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281836 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281839 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281841 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281844 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281846 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281848 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281851 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281854 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281856 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281859 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281861 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281864 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281866 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281869 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281871 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:07:16.286211 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281874 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:07:16.286707 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281876 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:07:16.286707 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281880 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:07:16.286707 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281883 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:07:16.286707 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281902 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:07:16.286707 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281907 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:07:16.286707 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281910 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:07:16.286707 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281913 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:07:16.286707 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281915 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:07:16.286707 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281918 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:07:16.286707 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281920 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:07:16.286707 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281923 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:07:16.286707 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281925 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:07:16.286707 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281928 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:07:16.286707 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281930 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:07:16.286707 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281934 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:07:16.286707 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281937 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:07:16.286707 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281940 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:07:16.286707 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281943 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:07:16.286707 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281945 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281948 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281950 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281953 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281956 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281959 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281961 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281964 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281967 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281971 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281973 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281976 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281978 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281981 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281983 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281986 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281988 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281991 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281993 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281996 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:07:16.287264 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.281998 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:07:16.287762 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.282000 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:07:16.287762 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.282003 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:07:16.287762 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.282005 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:07:16.287762 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.282007 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:07:16.287762 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.282010 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:07:16.287762 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.282012 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:07:16.287762 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.282014 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:07:16.287762 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.282017 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:07:16.287762 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:16.282019 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:07:16.287762 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.282024 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 23:07:16.287762 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.282133 2583 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 23:07:16.287762 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.285789 2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 23:07:16.287762 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.287330 2583 server.go:1019] "Starting client certificate rotation" Jul 02 23:07:16.287762 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.287430 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 23:07:16.288133 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.287467 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 23:07:16.290159 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.290145 2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 23:07:16.293085 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.293065 2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 23:07:16.305542 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.305520 2583 log.go:25] "Validated CRI v1 runtime API" Jul 02 23:07:16.307697 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.307680 2583 log.go:25] "Validated CRI v1 image API" Jul 02 23:07:16.308594 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.308579 2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 23:07:16.312329 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.312305 2583 fs.go:135] Filesystem UUIDs: map[370a7f5b-a96d-42ec-a5d9-1029f79a6a4d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 ed72b013-b0df-4113-a809-6877e13e59e8:/dev/nvme0n1p3] Jul 02 23:07:16.312404 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.312328 2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 23:07:16.317515 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.317402 2583 manager.go:216] Machine: {Timestamp:2026-07-02 23:07:16.316870624 +0000 UTC m=+0.339688728 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098287 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec233ac25e517414f7fab0bf3e8014d4 SystemUUID:ec233ac2-5e51-7414-f7fa-b0bf3e8014d4 BootID:04899697-fdf1-49fc-8af3-52f828966804 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9b:61:f1:7a:cb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9b:61:f1:7a:cb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:cf:5c:f6:13:65 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 23:07:16.317515 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.317506 2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 23:07:16.317628 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.317617 2583 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 23:07:16.318785 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.318767 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 23:07:16.319870 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.319846 2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 23:07:16.320053 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.319872 2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-243.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 23:07:16.321146 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.321135 2583 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 23:07:16.321179 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.321149 2583 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 23:07:16.321179 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.321156 2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 23:07:16.321725 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.321715 2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 23:07:16.322899 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.322880 2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 23:07:16.325611 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.325599 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 02 23:07:16.326105 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.326095 2583 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 23:07:16.326136 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.326128 2583 kubelet.go:486] "Attempting to sync node with API server" Jul 02 23:07:16.326170 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.326138 2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 23:07:16.326170 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.326153 2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 23:07:16.326170 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.326164 2583 kubelet.go:398] "Adding apiserver pod source" Jul 02 23:07:16.326255 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.326174 2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 23:07:16.328341 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.328113 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 23:07:16.328341 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.328145 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 23:07:16.330700 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.330678 2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 23:07:16.332454 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.332441 2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 23:07:16.332543 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.332471 2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 23:07:16.333483 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.333472 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 23:07:16.333523 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.333488 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 23:07:16.333523 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.333495 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 23:07:16.333523 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.333500 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 23:07:16.333523 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.333509 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 23:07:16.333523 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.333514 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 23:07:16.333523 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.333521 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 23:07:16.333699 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.333527 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 23:07:16.333699 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.333533 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 23:07:16.333699 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.333539 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 23:07:16.333699 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.333546 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 23:07:16.333699 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.333551 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 23:07:16.334384 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.334373 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 23:07:16.334384 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.334383 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 23:07:16.336127 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:16.336098 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-243.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 23:07:16.336200 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:16.336101 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 23:07:16.337815 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.337798 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hsrmf" Jul 02 23:07:16.337859 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.337829 2583 server.go:1268] "Started kubelet" Jul 02 23:07:16.337955 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.337925 2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 23:07:16.338048 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.337974 2583 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 23:07:16.338551 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.338517 2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 23:07:16.338737 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.338721 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-243.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 23:07:16.338833 ip-10-0-136-243 systemd[1]: Started Kubernetes Kubelet. Jul 02 23:07:16.339534 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.339519 2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 23:07:16.341106 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.341089 2583 server.go:310] "Adding debug handlers to kubelet server" Jul 02 23:07:16.343933 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:16.343905 2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 23:07:16.345136 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.345122 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hsrmf" Jul 02 23:07:16.345452 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.345428 2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 23:07:16.345537 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.345450 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 23:07:16.345703 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:16.345686 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-243.ec2.internal\" not found" Jul 02 23:07:16.346088 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.346071 2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 23:07:16.346088 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.346072 2583 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 23:07:16.346261 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.346098 2583 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 23:07:16.346261 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.346259 2583 reconstruct.go:97] "Volume reconstruction finished" Jul 02 23:07:16.346365 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.346270 2583 reconciler.go:29] "Reconciler: start to sync state" Jul 02 23:07:16.346670 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.346653 2583 factory.go:153] Registering CRI-O factory Jul 02 23:07:16.346670 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.346672 2583 factory.go:223] Registration of the crio container factory successfully Jul 02 23:07:16.346799 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.346722 2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 23:07:16.346799 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.346739 2583 factory.go:55] Registering systemd factory Jul 02 23:07:16.346799 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.346746 2583 factory.go:223] Registration of the systemd container factory successfully Jul 02 23:07:16.346799 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.346763 2583 factory.go:103] Registering Raw factory Jul 02 23:07:16.346799 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.346771 2583 manager.go:1196] Started watching for new ooms in manager Jul 02 23:07:16.347030 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:16.345970 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-243.ec2.internal.18be9ba5347cd86a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-243.ec2.internal,UID:ip-10-0-136-243.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-243.ec2.internal,},FirstTimestamp:2026-07-02 23:07:16.337801322 +0000 UTC m=+0.360619426,LastTimestamp:2026-07-02 23:07:16.337801322 +0000 UTC m=+0.360619426,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-243.ec2.internal,}" Jul 02 23:07:16.347348 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.347201 2583 manager.go:349] Starting recovery of all containers Jul 02 23:07:16.355375 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.355200 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:07:16.357784 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:16.357737 2583 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-243.ec2.internal\" not found" node="ip-10-0-136-243.ec2.internal" Jul 02 23:07:16.358016 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.357879 2583 manager.go:354] Recovery completed Jul 02 23:07:16.359790 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:16.359735 2583 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 02 23:07:16.362820 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.362808 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:07:16.364869 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.364851 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:07:16.364948 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.364899 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:07:16.364948 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.364913 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:07:16.365492 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.365479 2583 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 23:07:16.365538 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.365495 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:07:16.365538 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.365494 2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 23:07:16.365538 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.365537 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 02 23:07:16.366162 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.366148 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:07:16.366210 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.366178 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:07:16.366210 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.366189 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:07:16.366788 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.366777 2583 policy_none.go:49] "None policy: Start" Jul 02 23:07:16.366820 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.366793 2583 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 23:07:16.366820 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.366804 2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 23:07:16.367921 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.367911 2583 policy_none.go:47] "Start" Jul 02 23:07:16.417574 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.417557 2583 manager.go:337] "Starting Device Plugin manager" Jul 02 23:07:16.417678 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:16.417597 2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 23:07:16.417678 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.417610 2583 server.go:87] "Starting device plugin registration server" Jul 02 23:07:16.417930 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.417914 2583 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 23:07:16.417982 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.417931 2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 23:07:16.418094 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.418078 2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 23:07:16.418204 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.418156 2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 23:07:16.418204 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.418168 2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 23:07:16.418723 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:16.418703 2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 23:07:16.418812 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:16.418759 2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-243.ec2.internal\" not found" Jul 02 23:07:16.518559 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.518470 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:07:16.519705 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.519686 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:07:16.519780 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.519729 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:07:16.519780 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.519743 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:07:16.519780 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.519768 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-243.ec2.internal" Jul 02 23:07:16.524554 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.524531 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 23:07:16.525926 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.525881 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 23:07:16.526016 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.525951 2583 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 23:07:16.526016 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.525980 2583 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 23:07:16.526121 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:16.526023 2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 02 23:07:16.526908 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.526879 2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-243.ec2.internal" Jul 02 23:07:16.526995 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:16.526912 2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-243.ec2.internal\": node \"ip-10-0-136-243.ec2.internal\" not found" Jul 02 23:07:16.528728 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.528712 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:07:16.573345 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:16.573309 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-243.ec2.internal\" not found" Jul 02 23:07:16.626424 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.626383 2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-243.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal"] Jul 02 23:07:16.626522 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.626481 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:07:16.628186 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.628167 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:07:16.628275 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.628201 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:07:16.628275 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.628211 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:07:16.629791 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.629778 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:07:16.629865 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.629850 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-243.ec2.internal" Jul 02 23:07:16.629913 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.629904 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:07:16.630514 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.630495 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:07:16.630596 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.630528 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:07:16.630596 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.630541 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:07:16.630596 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.630499 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:07:16.630596 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.630595 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:07:16.630710 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.630604 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:07:16.631670 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.631655 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal" Jul 02 23:07:16.631757 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.631681 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:07:16.632333 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.632321 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:07:16.632398 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.632346 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:07:16.632398 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.632358 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:07:16.648630 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.648610 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e6bcbd2655d21ca4d5712d402679456b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal\" (UID: \"e6bcbd2655d21ca4d5712d402679456b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal" Jul 02 23:07:16.648700 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.648635 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6bcbd2655d21ca4d5712d402679456b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal\" (UID: \"e6bcbd2655d21ca4d5712d402679456b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal" Jul 02 23:07:16.648700 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.648653 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/20f20beb3812fd7a1ed1315ed6bb79ac-config\") pod \"kube-apiserver-proxy-ip-10-0-136-243.ec2.internal\" (UID: \"20f20beb3812fd7a1ed1315ed6bb79ac\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-243.ec2.internal" Jul 02 23:07:16.657531 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:16.657510 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-243.ec2.internal\" not found" node="ip-10-0-136-243.ec2.internal" Jul 02 23:07:16.662194 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:16.662179 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-243.ec2.internal\" not found" node="ip-10-0-136-243.ec2.internal" Jul 02 23:07:16.674233 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:16.674214 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-243.ec2.internal\" not found" Jul 02 23:07:16.749659 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.749632 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/20f20beb3812fd7a1ed1315ed6bb79ac-config\") pod \"kube-apiserver-proxy-ip-10-0-136-243.ec2.internal\" (UID: \"20f20beb3812fd7a1ed1315ed6bb79ac\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-243.ec2.internal" Jul 02 23:07:16.749743 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.749662 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e6bcbd2655d21ca4d5712d402679456b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal\" (UID: \"e6bcbd2655d21ca4d5712d402679456b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal" Jul 02 23:07:16.749743 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.749684 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6bcbd2655d21ca4d5712d402679456b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal\" (UID: \"e6bcbd2655d21ca4d5712d402679456b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal" Jul 02 23:07:16.749743 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.749712 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6bcbd2655d21ca4d5712d402679456b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal\" (UID: \"e6bcbd2655d21ca4d5712d402679456b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal" Jul 02 23:07:16.749743 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.749723 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/20f20beb3812fd7a1ed1315ed6bb79ac-config\") pod \"kube-apiserver-proxy-ip-10-0-136-243.ec2.internal\" (UID: \"20f20beb3812fd7a1ed1315ed6bb79ac\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-243.ec2.internal" Jul 02 23:07:16.749863 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.749723 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e6bcbd2655d21ca4d5712d402679456b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal\" (UID: \"e6bcbd2655d21ca4d5712d402679456b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal" Jul 02 23:07:16.774777 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:16.774719 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-243.ec2.internal\" not found" Jul 02 23:07:16.875758 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:16.875715 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-243.ec2.internal\" not found" Jul 02 23:07:16.960286 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.960261 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-243.ec2.internal" Jul 02 23:07:16.965189 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:16.965173 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal" Jul 02 23:07:16.975966 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:16.975942 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-243.ec2.internal\" not found" Jul 02 23:07:17.076578 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:17.076485 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-243.ec2.internal\" not found" Jul 02 23:07:17.176730 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:17.176690 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-243.ec2.internal\" not found" Jul 02 23:07:17.194696 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.194669 2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:07:17.246507 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.246469 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal" Jul 02 23:07:17.258774 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.258742 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 23:07:17.261071 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.261052 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-243.ec2.internal" Jul 02 23:07:17.268367 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.268344 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 23:07:17.286144 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.286117 2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 23:07:17.286298 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.286274 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 23:07:17.286366 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.286280 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 23:07:17.286366 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.286278 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 23:07:17.327522 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.327426 2583 apiserver.go:52] "Watching apiserver" Jul 02 23:07:17.346302 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.346268 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 23:07:17.347408 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.347382 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 23:02:17 +0000 UTC" deadline="2027-12-15 11:31:13.233006493 +0000 UTC" Jul 02 23:07:17.347408 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.347400 2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 23:07:17.347508 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.347409 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12732h23m55.885601749s" Jul 02 23:07:17.347749 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.347732 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-cnq78","openshift-network-diagnostics/network-check-target-drz87","openshift-ovn-kubernetes/ovnkube-node-fcv6k","kube-system/global-pull-secret-syncer-rs2g8","openshift-cluster-node-tuning-operator/tuned-hd6kw","openshift-image-registry/node-ca-bv9xn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal","openshift-multus/multus-additional-cni-plugins-xt994","openshift-multus/network-metrics-daemon-xqwp5","openshift-network-operator/iptables-alerter-xbnmr","kube-system/konnectivity-agent-ngwbb","kube-system/kube-apiserver-proxy-ip-10-0-136-243.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq","openshift-dns/node-resolver-b2nvt"] Jul 02 23:07:17.349373 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.349352 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xbnmr" Jul 02 23:07:17.351412 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.350648 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.353430 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.352074 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-h85zb\"" Jul 02 23:07:17.353430 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.352122 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:07:17.353430 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.352294 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 23:07:17.353430 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.352437 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 23:07:17.353430 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.352680 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.353759 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.353602 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 23:07:17.353759 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.353616 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 23:07:17.353759 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.353643 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 23:07:17.353759 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.353715 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-68nph\"" Jul 02 23:07:17.354915 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.354874 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:17.355040 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.355016 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 23:07:17.355108 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:17.355035 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rs2g8" podUID="dcb0135c-18a2-4e25-9193-16f4e2a3b833" Jul 02 23:07:17.355430 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.355381 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 23:07:17.355696 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.355681 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 23:07:17.355990 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.355957 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-m9kq9\"" Jul 02 23:07:17.355990 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.355957 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 23:07:17.356235 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.356218 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.356437 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.356269 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 23:07:17.356437 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.356224 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 23:07:17.357390 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.357370 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bv9xn" Jul 02 23:07:17.358423 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.358399 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:07:17.358603 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.358535 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dkdwc\"" Jul 02 23:07:17.358676 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.358635 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 23:07:17.358734 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.358695 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:17.358790 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:17.358759 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqwp5" podUID="e3f048d9-2670-488f-9a21-73f967c069b7" Jul 02 23:07:17.359619 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.359600 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 23:07:17.360144 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.359605 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 23:07:17.360144 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.359746 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 23:07:17.360144 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.359759 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 23:07:17.360144 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.359762 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qqxds\"" Jul 02 23:07:17.360144 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.360069 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b2nvt" Jul 02 23:07:17.361649 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.361630 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:17.361741 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:17.361696 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drz87" podUID="4431a02d-4597-4cb5-ad47-faaa53a10233" Jul 02 23:07:17.362376 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.362356 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 23:07:17.362463 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.362376 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nld5w\"" Jul 02 23:07:17.362513 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.362462 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 23:07:17.363350 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.363335 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.365465 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.365047 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ngwbb" Jul 02 23:07:17.365640 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.365605 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vlvr2\"" Jul 02 23:07:17.365640 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.365617 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 23:07:17.365819 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.365676 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 23:07:17.365819 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.365757 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 23:07:17.365819 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.365678 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 23:07:17.366549 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.366526 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.367364 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.367349 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 23:07:17.367776 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.367760 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 23:07:17.367892 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.367874 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cdxff\"" Jul 02 23:07:17.368755 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.368738 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 23:07:17.369053 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.369035 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8b927\"" Jul 02 23:07:17.369335 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.369322 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 23:07:17.394547 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.394521 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qdwgj" Jul 02 23:07:17.402858 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.402762 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qdwgj" Jul 02 23:07:17.447274 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.447245 2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 23:07:17.453407 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453377 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-systemd-units\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.453544 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453419 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d952dcf9-8a86-487d-9cfb-b6c592a17982-multus-daemon-config\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.453544 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453447 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-host-run-multus-certs\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.453544 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453482 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a3a17f85-d07c-4017-935c-fb5dfe60dec9-agent-certs\") pod \"konnectivity-agent-ngwbb\" (UID: \"a3a17f85-d07c-4017-935c-fb5dfe60dec9\") " pod="kube-system/konnectivity-agent-ngwbb" Jul 02 23:07:17.453544 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453505 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs\") pod \"network-metrics-daemon-xqwp5\" (UID: \"e3f048d9-2670-488f-9a21-73f967c069b7\") " pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:17.453544 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453524 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-run-netns\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.453792 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453553 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-etc-systemd\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.453792 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453578 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2055d544-3273-4518-b065-e41ceba96443-host\") pod \"node-ca-bv9xn\" (UID: \"2055d544-3273-4518-b065-e41ceba96443\") " pod="openshift-image-registry/node-ca-bv9xn" Jul 02 23:07:17.453792 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453601 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-system-cni-dir\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.453792 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453624 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-host-var-lib-kubelet\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.453792 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453650 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e98ff5d8-d7cf-4864-a5f1-a2eefcde889c-iptables-alerter-script\") pod \"iptables-alerter-xbnmr\" (UID: \"e98ff5d8-d7cf-4864-a5f1-a2eefcde889c\") " pod="openshift-network-operator/iptables-alerter-xbnmr" Jul 02 23:07:17.453792 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453686 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad87676b-0e54-4723-895d-9e19afedfca7-tmp-dir\") pod \"node-resolver-b2nvt\" (UID: \"ad87676b-0e54-4723-895d-9e19afedfca7\") " pod="openshift-dns/node-resolver-b2nvt" Jul 02 23:07:17.453792 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453712 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/511288f9-603d-46d3-aaf1-f1725e8f3226-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.453792 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453736 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-etc-modprobe-d\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.453792 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453760 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkxpv\" (UniqueName: \"kubernetes.io/projected/d952dcf9-8a86-487d-9cfb-b6c592a17982-kube-api-access-lkxpv\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.453792 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453793 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57aefc7d-7069-45b4-9f4d-4612bebd980d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.454202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453859 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.454202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453908 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-sys\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.454202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453929 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a3a17f85-d07c-4017-935c-fb5dfe60dec9-konnectivity-ca\") pod \"konnectivity-agent-ngwbb\" (UID: \"a3a17f85-d07c-4017-935c-fb5dfe60dec9\") " pod="kube-system/konnectivity-agent-ngwbb" Jul 02 23:07:17.454202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453948 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-cni-bin\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.454202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453963 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-multus-cni-dir\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.454202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453978 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-os-release\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.454202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.453993 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d952dcf9-8a86-487d-9cfb-b6c592a17982-cni-binary-copy\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.454202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454014 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-host-run-netns\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.454202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454029 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-host-var-lib-cni-bin\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.454202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454044 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-run-openvswitch\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.454202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454065 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j25q7\" (UniqueName: \"kubernetes.io/projected/2055d544-3273-4518-b065-e41ceba96443-kube-api-access-j25q7\") pod \"node-ca-bv9xn\" (UID: \"2055d544-3273-4518-b065-e41ceba96443\") " pod="openshift-image-registry/node-ca-bv9xn" Jul 02 23:07:17.454202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454091 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-etc-kubernetes\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.454202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454110 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57aefc7d-7069-45b4-9f4d-4612bebd980d-device-dir\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.454202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454125 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-etc-openvswitch\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.454202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454146 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-node-log\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.454202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454169 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dcb0135c-18a2-4e25-9193-16f4e2a3b833-dbus\") pod \"global-pull-secret-syncer-rs2g8\" (UID: \"dcb0135c-18a2-4e25-9193-16f4e2a3b833\") " pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:17.454202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454191 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-etc-kubernetes\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.454935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454208 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-etc-sysctl-d\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.454935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454230 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/511288f9-603d-46d3-aaf1-f1725e8f3226-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.454935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454256 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-cni-netd\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.454935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454277 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret\") pod \"global-pull-secret-syncer-rs2g8\" (UID: \"dcb0135c-18a2-4e25-9193-16f4e2a3b833\") " pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:17.454935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454292 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b172564e-817c-4344-b401-44f4322766f1-etc-tuned\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.454935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454308 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57aefc7d-7069-45b4-9f4d-4612bebd980d-socket-dir\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.454935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454330 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfq7m\" (UniqueName: \"kubernetes.io/projected/57aefc7d-7069-45b4-9f4d-4612bebd980d-kube-api-access-pfq7m\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.454935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454369 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-log-socket\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.454935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454407 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-run\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.454935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454439 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-multus-socket-dir-parent\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.454935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454467 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb571463-b959-4b84-bba1-2b22b22157fd-ovn-node-metrics-cert\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.454935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454495 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8dz4\" (UniqueName: \"kubernetes.io/projected/e98ff5d8-d7cf-4864-a5f1-a2eefcde889c-kube-api-access-j8dz4\") pod \"iptables-alerter-xbnmr\" (UID: \"e98ff5d8-d7cf-4864-a5f1-a2eefcde889c\") " pod="openshift-network-operator/iptables-alerter-xbnmr" Jul 02 23:07:17.454935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454520 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57aefc7d-7069-45b4-9f4d-4612bebd980d-etc-selinux\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.454935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454547 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/511288f9-603d-46d3-aaf1-f1725e8f3226-cnibin\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.454935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454570 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-etc-sysctl-conf\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.454935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454594 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22qjs\" (UniqueName: \"kubernetes.io/projected/b172564e-817c-4344-b401-44f4322766f1-kube-api-access-22qjs\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.455636 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454621 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-host-run-k8s-cni-cncf-io\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.455636 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454645 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-host-var-lib-cni-multus\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.455636 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454670 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57aefc7d-7069-45b4-9f4d-4612bebd980d-registration-dir\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.455636 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454695 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-etc-sysconfig\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.455636 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454718 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-host\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.455636 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454753 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-multus-conf-dir\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.455636 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454787 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ad87676b-0e54-4723-895d-9e19afedfca7-hosts-file\") pod \"node-resolver-b2nvt\" (UID: \"ad87676b-0e54-4723-895d-9e19afedfca7\") " pod="openshift-dns/node-resolver-b2nvt" Jul 02 23:07:17.455636 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454817 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdx6l\" (UniqueName: \"kubernetes.io/projected/ad87676b-0e54-4723-895d-9e19afedfca7-kube-api-access-xdx6l\") pod \"node-resolver-b2nvt\" (UID: \"ad87676b-0e54-4723-895d-9e19afedfca7\") " pod="openshift-dns/node-resolver-b2nvt" Jul 02 23:07:17.455636 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454874 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtgl7\" (UniqueName: \"kubernetes.io/projected/511288f9-603d-46d3-aaf1-f1725e8f3226-kube-api-access-gtgl7\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.455636 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454924 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-kubelet\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.455636 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454951 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-var-lib-kubelet\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.455636 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.454976 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/511288f9-603d-46d3-aaf1-f1725e8f3226-system-cni-dir\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.455636 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455003 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/511288f9-603d-46d3-aaf1-f1725e8f3226-os-release\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.455636 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455026 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-run-systemd\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.455636 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455047 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-run-ovn\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.455636 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455068 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b172564e-817c-4344-b401-44f4322766f1-tmp\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.456276 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455089 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-hostroot\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.456276 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455115 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e98ff5d8-d7cf-4864-a5f1-a2eefcde889c-host-slash\") pod \"iptables-alerter-xbnmr\" (UID: \"e98ff5d8-d7cf-4864-a5f1-a2eefcde889c\") " pod="openshift-network-operator/iptables-alerter-xbnmr" Jul 02 23:07:17.456276 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455138 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/511288f9-603d-46d3-aaf1-f1725e8f3226-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.456276 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455176 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-slash\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.456276 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455200 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-var-lib-openvswitch\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.456276 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455225 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-run-ovn-kubernetes\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.456276 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455248 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cb571463-b959-4b84-bba1-2b22b22157fd-ovnkube-script-lib\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.456276 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455273 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57aefc7d-7069-45b4-9f4d-4612bebd980d-sys-fs\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.456276 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455356 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cb571463-b959-4b84-bba1-2b22b22157fd-ovnkube-config\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.456276 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455379 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cb571463-b959-4b84-bba1-2b22b22157fd-env-overrides\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.456276 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455399 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6wnb\" (UniqueName: \"kubernetes.io/projected/cb571463-b959-4b84-bba1-2b22b22157fd-kube-api-access-v6wnb\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.456276 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455423 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dcb0135c-18a2-4e25-9193-16f4e2a3b833-kubelet-config\") pod \"global-pull-secret-syncer-rs2g8\" (UID: \"dcb0135c-18a2-4e25-9193-16f4e2a3b833\") " pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:17.456276 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455463 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-lib-modules\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.456276 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455510 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2055d544-3273-4518-b065-e41ceba96443-serviceca\") pod \"node-ca-bv9xn\" (UID: \"2055d544-3273-4518-b065-e41ceba96443\") " pod="openshift-image-registry/node-ca-bv9xn" Jul 02 23:07:17.456276 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455537 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-cnibin\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.456276 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455563 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvj5n\" (UniqueName: \"kubernetes.io/projected/e3f048d9-2670-488f-9a21-73f967c069b7-kube-api-access-fvj5n\") pod \"network-metrics-daemon-xqwp5\" (UID: \"e3f048d9-2670-488f-9a21-73f967c069b7\") " pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:17.456955 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455602 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzfdj\" (UniqueName: \"kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj\") pod \"network-check-target-drz87\" (UID: \"4431a02d-4597-4cb5-ad47-faaa53a10233\") " pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:17.456955 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.455648 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/511288f9-603d-46d3-aaf1-f1725e8f3226-cni-binary-copy\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.524597 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:17.524561 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6bcbd2655d21ca4d5712d402679456b.slice/crio-20183d9b6bec733709ac9ccf69d85c46cad743e4025ab942d15b4e941e280659 WatchSource:0}: Error finding container 20183d9b6bec733709ac9ccf69d85c46cad743e4025ab942d15b4e941e280659: Status 404 returned error can't find the container with id 20183d9b6bec733709ac9ccf69d85c46cad743e4025ab942d15b4e941e280659 Jul 02 23:07:17.524771 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:17.524745 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f20beb3812fd7a1ed1315ed6bb79ac.slice/crio-a4c36963c1c6170482411601bd8810d51f257b86d2ca48f9edc1fc21c79367b2 WatchSource:0}: Error finding container a4c36963c1c6170482411601bd8810d51f257b86d2ca48f9edc1fc21c79367b2: Status 404 returned error can't find the container with id a4c36963c1c6170482411601bd8810d51f257b86d2ca48f9edc1fc21c79367b2 Jul 02 23:07:17.531081 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.531066 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:07:17.557072 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557044 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-lib-modules\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.557072 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557076 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2055d544-3273-4518-b065-e41ceba96443-serviceca\") pod \"node-ca-bv9xn\" (UID: \"2055d544-3273-4518-b065-e41ceba96443\") " pod="openshift-image-registry/node-ca-bv9xn" Jul 02 23:07:17.557390 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557093 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-cnibin\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.557390 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557111 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvj5n\" (UniqueName: \"kubernetes.io/projected/e3f048d9-2670-488f-9a21-73f967c069b7-kube-api-access-fvj5n\") pod \"network-metrics-daemon-xqwp5\" (UID: \"e3f048d9-2670-488f-9a21-73f967c069b7\") " pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:17.557390 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557129 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzfdj\" (UniqueName: \"kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj\") pod \"network-check-target-drz87\" (UID: \"4431a02d-4597-4cb5-ad47-faaa53a10233\") " pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:17.557390 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557145 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/511288f9-603d-46d3-aaf1-f1725e8f3226-cni-binary-copy\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.557390 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557162 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-systemd-units\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.557390 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557180 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d952dcf9-8a86-487d-9cfb-b6c592a17982-multus-daemon-config\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.557390 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557207 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-host-run-multus-certs\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.557390 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557232 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a3a17f85-d07c-4017-935c-fb5dfe60dec9-agent-certs\") pod \"konnectivity-agent-ngwbb\" (UID: \"a3a17f85-d07c-4017-935c-fb5dfe60dec9\") " pod="kube-system/konnectivity-agent-ngwbb" Jul 02 23:07:17.557390 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557255 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs\") pod \"network-metrics-daemon-xqwp5\" (UID: \"e3f048d9-2670-488f-9a21-73f967c069b7\") " pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:17.557390 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557281 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-run-netns\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.557390 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557285 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-cnibin\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.557390 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557307 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-etc-systemd\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.557390 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557285 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-lib-modules\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.557390 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557330 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2055d544-3273-4518-b065-e41ceba96443-host\") pod \"node-ca-bv9xn\" (UID: \"2055d544-3273-4518-b065-e41ceba96443\") " pod="openshift-image-registry/node-ca-bv9xn" Jul 02 23:07:17.557390 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557355 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-system-cni-dir\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.557390 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557379 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-host-var-lib-kubelet\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.558214 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557404 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e98ff5d8-d7cf-4864-a5f1-a2eefcde889c-iptables-alerter-script\") pod \"iptables-alerter-xbnmr\" (UID: \"e98ff5d8-d7cf-4864-a5f1-a2eefcde889c\") " pod="openshift-network-operator/iptables-alerter-xbnmr" Jul 02 23:07:17.558214 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:17.557415 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:07:17.558214 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557433 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad87676b-0e54-4723-895d-9e19afedfca7-tmp-dir\") pod \"node-resolver-b2nvt\" (UID: \"ad87676b-0e54-4723-895d-9e19afedfca7\") " pod="openshift-dns/node-resolver-b2nvt" Jul 02 23:07:17.558214 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557457 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/511288f9-603d-46d3-aaf1-f1725e8f3226-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.558214 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:17.557494 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs podName:e3f048d9-2670-488f-9a21-73f967c069b7 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:18.05747211 +0000 UTC m=+2.080290206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs") pod "network-metrics-daemon-xqwp5" (UID: "e3f048d9-2670-488f-9a21-73f967c069b7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:07:17.558214 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557522 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-etc-modprobe-d\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.558214 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557521 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-systemd-units\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.558214 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557555 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkxpv\" (UniqueName: \"kubernetes.io/projected/d952dcf9-8a86-487d-9cfb-b6c592a17982-kube-api-access-lkxpv\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.558214 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557582 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57aefc7d-7069-45b4-9f4d-4612bebd980d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.558214 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557586 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-run-netns\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.558214 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557608 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.558214 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557622 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/511288f9-603d-46d3-aaf1-f1725e8f3226-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.558214 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557634 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-sys\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.558214 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557641 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-etc-systemd\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.558214 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557659 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a3a17f85-d07c-4017-935c-fb5dfe60dec9-konnectivity-ca\") pod \"konnectivity-agent-ngwbb\" (UID: \"a3a17f85-d07c-4017-935c-fb5dfe60dec9\") " pod="kube-system/konnectivity-agent-ngwbb" Jul 02 23:07:17.558214 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557683 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-cni-bin\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.558214 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557685 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-system-cni-dir\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.559153 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557686 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2055d544-3273-4518-b065-e41ceba96443-host\") pod \"node-ca-bv9xn\" (UID: \"2055d544-3273-4518-b065-e41ceba96443\") " pod="openshift-image-registry/node-ca-bv9xn" Jul 02 23:07:17.559153 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557698 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2055d544-3273-4518-b065-e41ceba96443-serviceca\") pod \"node-ca-bv9xn\" (UID: \"2055d544-3273-4518-b065-e41ceba96443\") " pod="openshift-image-registry/node-ca-bv9xn" Jul 02 23:07:17.559153 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557737 2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 23:07:17.559153 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557777 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-multus-cni-dir\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.559153 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557822 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-host-var-lib-kubelet\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.559153 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557828 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/511288f9-603d-46d3-aaf1-f1725e8f3226-cni-binary-copy\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.559153 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557825 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-etc-modprobe-d\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.559153 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557329 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-host-run-multus-certs\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.559153 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557866 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57aefc7d-7069-45b4-9f4d-4612bebd980d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.559153 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557914 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.559153 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557927 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-sys\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.559153 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557943 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d952dcf9-8a86-487d-9cfb-b6c592a17982-multus-daemon-config\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.559153 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557967 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-cni-bin\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.559153 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.557708 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-multus-cni-dir\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.559153 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558084 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad87676b-0e54-4723-895d-9e19afedfca7-tmp-dir\") pod \"node-resolver-b2nvt\" (UID: \"ad87676b-0e54-4723-895d-9e19afedfca7\") " pod="openshift-dns/node-resolver-b2nvt" Jul 02 23:07:17.559153 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558121 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-os-release\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.559153 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558152 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d952dcf9-8a86-487d-9cfb-b6c592a17982-cni-binary-copy\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.559153 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558179 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-host-run-netns\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.560099 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558208 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-host-var-lib-cni-bin\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.560099 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558216 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-os-release\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.560099 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558234 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-run-openvswitch\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.560099 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558263 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j25q7\" (UniqueName: \"kubernetes.io/projected/2055d544-3273-4518-b065-e41ceba96443-kube-api-access-j25q7\") pod \"node-ca-bv9xn\" (UID: \"2055d544-3273-4518-b065-e41ceba96443\") " pod="openshift-image-registry/node-ca-bv9xn" Jul 02 23:07:17.560099 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558277 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-host-var-lib-cni-bin\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.560099 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558282 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-host-run-netns\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.560099 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558289 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-etc-kubernetes\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.560099 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558324 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-run-openvswitch\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.560099 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558322 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e98ff5d8-d7cf-4864-a5f1-a2eefcde889c-iptables-alerter-script\") pod \"iptables-alerter-xbnmr\" (UID: \"e98ff5d8-d7cf-4864-a5f1-a2eefcde889c\") " pod="openshift-network-operator/iptables-alerter-xbnmr" Jul 02 23:07:17.560099 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558333 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-etc-kubernetes\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.560099 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558386 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57aefc7d-7069-45b4-9f4d-4612bebd980d-device-dir\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.560099 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558406 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-etc-openvswitch\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.560099 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558422 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-node-log\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.560099 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558438 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dcb0135c-18a2-4e25-9193-16f4e2a3b833-dbus\") pod \"global-pull-secret-syncer-rs2g8\" (UID: \"dcb0135c-18a2-4e25-9193-16f4e2a3b833\") " pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:17.560099 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558457 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-etc-kubernetes\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.560099 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558482 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-etc-sysctl-d\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.560099 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558508 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/511288f9-603d-46d3-aaf1-f1725e8f3226-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.560904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558525 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57aefc7d-7069-45b4-9f4d-4612bebd980d-device-dir\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.560904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558510 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a3a17f85-d07c-4017-935c-fb5dfe60dec9-konnectivity-ca\") pod \"konnectivity-agent-ngwbb\" (UID: \"a3a17f85-d07c-4017-935c-fb5dfe60dec9\") " pod="kube-system/konnectivity-agent-ngwbb" Jul 02 23:07:17.560904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558538 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-cni-netd\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.560904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558564 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-node-log\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.560904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558510 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-etc-openvswitch\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.560904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558589 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-cni-netd\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.560904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558592 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret\") pod \"global-pull-secret-syncer-rs2g8\" (UID: \"dcb0135c-18a2-4e25-9193-16f4e2a3b833\") " pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:17.560904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558629 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dcb0135c-18a2-4e25-9193-16f4e2a3b833-dbus\") pod \"global-pull-secret-syncer-rs2g8\" (UID: \"dcb0135c-18a2-4e25-9193-16f4e2a3b833\") " pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:17.560904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558636 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-etc-kubernetes\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.560904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558639 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b172564e-817c-4344-b401-44f4322766f1-etc-tuned\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.560904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558637 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-etc-sysctl-d\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.560904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558692 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d952dcf9-8a86-487d-9cfb-b6c592a17982-cni-binary-copy\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.560904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558715 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57aefc7d-7069-45b4-9f4d-4612bebd980d-socket-dir\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.560904 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:17.558727 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:07:17.560904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558739 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfq7m\" (UniqueName: \"kubernetes.io/projected/57aefc7d-7069-45b4-9f4d-4612bebd980d-kube-api-access-pfq7m\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.560904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558758 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-log-socket\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.560904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558773 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-run\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.561753 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:17.558804 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret podName:dcb0135c-18a2-4e25-9193-16f4e2a3b833 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:18.058787342 +0000 UTC m=+2.081605434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret") pod "global-pull-secret-syncer-rs2g8" (UID: "dcb0135c-18a2-4e25-9193-16f4e2a3b833") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:07:17.561753 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558824 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-run\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.561753 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558825 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-log-socket\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.561753 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558896 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57aefc7d-7069-45b4-9f4d-4612bebd980d-socket-dir\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.561753 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558943 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-multus-socket-dir-parent\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.561753 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558967 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb571463-b959-4b84-bba1-2b22b22157fd-ovn-node-metrics-cert\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.561753 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.558995 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8dz4\" (UniqueName: \"kubernetes.io/projected/e98ff5d8-d7cf-4864-a5f1-a2eefcde889c-kube-api-access-j8dz4\") pod \"iptables-alerter-xbnmr\" (UID: \"e98ff5d8-d7cf-4864-a5f1-a2eefcde889c\") " pod="openshift-network-operator/iptables-alerter-xbnmr" Jul 02 23:07:17.561753 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559018 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-multus-socket-dir-parent\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.561753 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559023 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57aefc7d-7069-45b4-9f4d-4612bebd980d-etc-selinux\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.561753 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559091 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/511288f9-603d-46d3-aaf1-f1725e8f3226-cnibin\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.561753 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559111 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/511288f9-603d-46d3-aaf1-f1725e8f3226-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.561753 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559172 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57aefc7d-7069-45b4-9f4d-4612bebd980d-etc-selinux\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.561753 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559241 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-etc-sysctl-conf\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.561753 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559251 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/511288f9-603d-46d3-aaf1-f1725e8f3226-cnibin\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.561753 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559271 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22qjs\" (UniqueName: \"kubernetes.io/projected/b172564e-817c-4344-b401-44f4322766f1-kube-api-access-22qjs\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.561753 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559362 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-etc-sysctl-conf\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.562402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559497 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-host-run-k8s-cni-cncf-io\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.562402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559543 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-host-var-lib-cni-multus\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.562402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559601 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-host-var-lib-cni-multus\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.562402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559669 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57aefc7d-7069-45b4-9f4d-4612bebd980d-registration-dir\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.562402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559691 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-etc-sysconfig\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.562402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559712 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-host\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.562402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559737 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-multus-conf-dir\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.562402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559755 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57aefc7d-7069-45b4-9f4d-4612bebd980d-registration-dir\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.562402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559761 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-etc-sysconfig\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.562402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559774 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-host-run-k8s-cni-cncf-io\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.562402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559763 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ad87676b-0e54-4723-895d-9e19afedfca7-hosts-file\") pod \"node-resolver-b2nvt\" (UID: \"ad87676b-0e54-4723-895d-9e19afedfca7\") " pod="openshift-dns/node-resolver-b2nvt" Jul 02 23:07:17.562402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559807 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-host\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.562402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559817 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdx6l\" (UniqueName: \"kubernetes.io/projected/ad87676b-0e54-4723-895d-9e19afedfca7-kube-api-access-xdx6l\") pod \"node-resolver-b2nvt\" (UID: \"ad87676b-0e54-4723-895d-9e19afedfca7\") " pod="openshift-dns/node-resolver-b2nvt" Jul 02 23:07:17.562402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559848 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ad87676b-0e54-4723-895d-9e19afedfca7-hosts-file\") pod \"node-resolver-b2nvt\" (UID: \"ad87676b-0e54-4723-895d-9e19afedfca7\") " pod="openshift-dns/node-resolver-b2nvt" Jul 02 23:07:17.562402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559850 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-multus-conf-dir\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.562402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559886 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtgl7\" (UniqueName: \"kubernetes.io/projected/511288f9-603d-46d3-aaf1-f1725e8f3226-kube-api-access-gtgl7\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.562402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.559931 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-kubelet\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.562402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560003 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-var-lib-kubelet\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.562945 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560020 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-kubelet\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.562945 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560101 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/511288f9-603d-46d3-aaf1-f1725e8f3226-system-cni-dir\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.562945 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560131 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b172564e-817c-4344-b401-44f4322766f1-var-lib-kubelet\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.562945 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560192 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/511288f9-603d-46d3-aaf1-f1725e8f3226-os-release\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.562945 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560232 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/511288f9-603d-46d3-aaf1-f1725e8f3226-system-cni-dir\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.562945 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560228 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-run-systemd\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.562945 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560271 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-run-ovn\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.562945 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560276 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/511288f9-603d-46d3-aaf1-f1725e8f3226-os-release\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.562945 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560290 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b172564e-817c-4344-b401-44f4322766f1-tmp\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.562945 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560327 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-run-systemd\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.562945 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560350 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-run-ovn\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.562945 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560356 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-hostroot\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.562945 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560387 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d952dcf9-8a86-487d-9cfb-b6c592a17982-hostroot\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.562945 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560390 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e98ff5d8-d7cf-4864-a5f1-a2eefcde889c-host-slash\") pod \"iptables-alerter-xbnmr\" (UID: \"e98ff5d8-d7cf-4864-a5f1-a2eefcde889c\") " pod="openshift-network-operator/iptables-alerter-xbnmr" Jul 02 23:07:17.562945 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560417 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/511288f9-603d-46d3-aaf1-f1725e8f3226-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.562945 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560436 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e98ff5d8-d7cf-4864-a5f1-a2eefcde889c-host-slash\") pod \"iptables-alerter-xbnmr\" (UID: \"e98ff5d8-d7cf-4864-a5f1-a2eefcde889c\") " pod="openshift-network-operator/iptables-alerter-xbnmr" Jul 02 23:07:17.562945 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560443 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-slash\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.563459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560469 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-var-lib-openvswitch\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.563459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560496 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-run-ovn-kubernetes\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.563459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560523 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cb571463-b959-4b84-bba1-2b22b22157fd-ovnkube-script-lib\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.563459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560531 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-slash\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.563459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560549 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57aefc7d-7069-45b4-9f4d-4612bebd980d-sys-fs\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.563459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560569 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-var-lib-openvswitch\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.563459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560573 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cb571463-b959-4b84-bba1-2b22b22157fd-ovnkube-config\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.563459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560615 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cb571463-b959-4b84-bba1-2b22b22157fd-env-overrides\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.563459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560644 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6wnb\" (UniqueName: \"kubernetes.io/projected/cb571463-b959-4b84-bba1-2b22b22157fd-kube-api-access-v6wnb\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.563459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560665 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57aefc7d-7069-45b4-9f4d-4612bebd980d-sys-fs\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.563459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560675 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dcb0135c-18a2-4e25-9193-16f4e2a3b833-kubelet-config\") pod \"global-pull-secret-syncer-rs2g8\" (UID: \"dcb0135c-18a2-4e25-9193-16f4e2a3b833\") " pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:17.563459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560585 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb571463-b959-4b84-bba1-2b22b22157fd-host-run-ovn-kubernetes\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.563459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.560974 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dcb0135c-18a2-4e25-9193-16f4e2a3b833-kubelet-config\") pod \"global-pull-secret-syncer-rs2g8\" (UID: \"dcb0135c-18a2-4e25-9193-16f4e2a3b833\") " pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:17.563459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.561009 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/511288f9-603d-46d3-aaf1-f1725e8f3226-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.563459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.561222 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cb571463-b959-4b84-bba1-2b22b22157fd-ovnkube-config\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.563459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.561624 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cb571463-b959-4b84-bba1-2b22b22157fd-ovnkube-script-lib\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.563459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.561642 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cb571463-b959-4b84-bba1-2b22b22157fd-env-overrides\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.564005 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.561680 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b172564e-817c-4344-b401-44f4322766f1-etc-tuned\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.564005 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.561863 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb571463-b959-4b84-bba1-2b22b22157fd-ovn-node-metrics-cert\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.564005 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.562083 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a3a17f85-d07c-4017-935c-fb5dfe60dec9-agent-certs\") pod \"konnectivity-agent-ngwbb\" (UID: \"a3a17f85-d07c-4017-935c-fb5dfe60dec9\") " pod="kube-system/konnectivity-agent-ngwbb" Jul 02 23:07:17.564005 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.562715 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b172564e-817c-4344-b401-44f4322766f1-tmp\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.567376 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:17.567357 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:07:17.567376 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:17.567377 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:07:17.567513 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:17.567388 2583 projected.go:196] Error preparing data for projected volume kube-api-access-kzfdj for pod openshift-network-diagnostics/network-check-target-drz87: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:07:17.567513 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:17.567448 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj podName:4431a02d-4597-4cb5-ad47-faaa53a10233 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:18.067434447 +0000 UTC m=+2.090252545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kzfdj" (UniqueName: "kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj") pod "network-check-target-drz87" (UID: "4431a02d-4597-4cb5-ad47-faaa53a10233") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:07:17.571938 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.571914 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j25q7\" (UniqueName: \"kubernetes.io/projected/2055d544-3273-4518-b065-e41ceba96443-kube-api-access-j25q7\") pod \"node-ca-bv9xn\" (UID: \"2055d544-3273-4518-b065-e41ceba96443\") " pod="openshift-image-registry/node-ca-bv9xn" Jul 02 23:07:17.573053 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.572166 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtgl7\" (UniqueName: \"kubernetes.io/projected/511288f9-603d-46d3-aaf1-f1725e8f3226-kube-api-access-gtgl7\") pod \"multus-additional-cni-plugins-xt994\" (UID: \"511288f9-603d-46d3-aaf1-f1725e8f3226\") " pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.573053 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.572282 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8dz4\" (UniqueName: \"kubernetes.io/projected/e98ff5d8-d7cf-4864-a5f1-a2eefcde889c-kube-api-access-j8dz4\") pod \"iptables-alerter-xbnmr\" (UID: \"e98ff5d8-d7cf-4864-a5f1-a2eefcde889c\") " pod="openshift-network-operator/iptables-alerter-xbnmr" Jul 02 23:07:17.573053 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.572500 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfq7m\" (UniqueName: \"kubernetes.io/projected/57aefc7d-7069-45b4-9f4d-4612bebd980d-kube-api-access-pfq7m\") pod \"aws-ebs-csi-driver-node-tdgmq\" (UID: \"57aefc7d-7069-45b4-9f4d-4612bebd980d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.573053 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.573010 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvj5n\" (UniqueName: \"kubernetes.io/projected/e3f048d9-2670-488f-9a21-73f967c069b7-kube-api-access-fvj5n\") pod \"network-metrics-daemon-xqwp5\" (UID: \"e3f048d9-2670-488f-9a21-73f967c069b7\") " pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:17.573548 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.573522 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdx6l\" (UniqueName: \"kubernetes.io/projected/ad87676b-0e54-4723-895d-9e19afedfca7-kube-api-access-xdx6l\") pod \"node-resolver-b2nvt\" (UID: \"ad87676b-0e54-4723-895d-9e19afedfca7\") " pod="openshift-dns/node-resolver-b2nvt" Jul 02 23:07:17.573811 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.573794 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6wnb\" (UniqueName: \"kubernetes.io/projected/cb571463-b959-4b84-bba1-2b22b22157fd-kube-api-access-v6wnb\") pod \"ovnkube-node-fcv6k\" (UID: \"cb571463-b959-4b84-bba1-2b22b22157fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.574047 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.574031 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkxpv\" (UniqueName: \"kubernetes.io/projected/d952dcf9-8a86-487d-9cfb-b6c592a17982-kube-api-access-lkxpv\") pod \"multus-cnq78\" (UID: \"d952dcf9-8a86-487d-9cfb-b6c592a17982\") " pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.574219 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.574202 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22qjs\" (UniqueName: \"kubernetes.io/projected/b172564e-817c-4344-b401-44f4322766f1-kube-api-access-22qjs\") pod \"tuned-hd6kw\" (UID: \"b172564e-817c-4344-b401-44f4322766f1\") " pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.671020 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.670914 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xbnmr" Jul 02 23:07:17.671828 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.671811 2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:07:17.680643 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:17.680619 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode98ff5d8_d7cf_4864_a5f1_a2eefcde889c.slice/crio-67921577b085b2943a0c6f337fec8d415a1ab097813e4f99a2ff198335a5de68 WatchSource:0}: Error finding container 67921577b085b2943a0c6f337fec8d415a1ab097813e4f99a2ff198335a5de68: Status 404 returned error can't find the container with id 67921577b085b2943a0c6f337fec8d415a1ab097813e4f99a2ff198335a5de68 Jul 02 23:07:17.686052 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.686032 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b2nvt" Jul 02 23:07:17.693827 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:17.693805 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad87676b_0e54_4723_895d_9e19afedfca7.slice/crio-febea4275d51784a99b206c300800709a71c35bd05692d313ae0465fbd5d9db7 WatchSource:0}: Error finding container febea4275d51784a99b206c300800709a71c35bd05692d313ae0465fbd5d9db7: Status 404 returned error can't find the container with id febea4275d51784a99b206c300800709a71c35bd05692d313ae0465fbd5d9db7 Jul 02 23:07:17.700736 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.700715 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" Jul 02 23:07:17.708617 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:17.708590 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57aefc7d_7069_45b4_9f4d_4612bebd980d.slice/crio-e67d6425a744c50738c47db46592b791d79c6581b0e5b15805400003a0b1242b WatchSource:0}: Error finding container e67d6425a744c50738c47db46592b791d79c6581b0e5b15805400003a0b1242b: Status 404 returned error can't find the container with id e67d6425a744c50738c47db46592b791d79c6581b0e5b15805400003a0b1242b Jul 02 23:07:17.712919 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.712896 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:17.720275 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:17.720250 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb571463_b959_4b84_bba1_2b22b22157fd.slice/crio-e90bd637d2f47945a015fef0b2cf92ec3ad33b63ebc81a7fbdb14b242dec0969 WatchSource:0}: Error finding container e90bd637d2f47945a015fef0b2cf92ec3ad33b63ebc81a7fbdb14b242dec0969: Status 404 returned error can't find the container with id e90bd637d2f47945a015fef0b2cf92ec3ad33b63ebc81a7fbdb14b242dec0969 Jul 02 23:07:17.729081 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.729063 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" Jul 02 23:07:17.736735 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:17.736710 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb172564e_817c_4344_b401_44f4322766f1.slice/crio-d724db865f8f43cf5b4dc0b55e12b2169fc4d87e80c4b71f1e37a05ff12eddd5 WatchSource:0}: Error finding container d724db865f8f43cf5b4dc0b55e12b2169fc4d87e80c4b71f1e37a05ff12eddd5: Status 404 returned error can't find the container with id d724db865f8f43cf5b4dc0b55e12b2169fc4d87e80c4b71f1e37a05ff12eddd5 Jul 02 23:07:17.751996 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.751975 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bv9xn" Jul 02 23:07:17.759708 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:17.759686 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2055d544_3273_4518_b065_e41ceba96443.slice/crio-ddb1d56f8dfc01f84ec1053309c678b8ab50815e2b973d422f3d715f0a4bb59f WatchSource:0}: Error finding container ddb1d56f8dfc01f84ec1053309c678b8ab50815e2b973d422f3d715f0a4bb59f: Status 404 returned error can't find the container with id ddb1d56f8dfc01f84ec1053309c678b8ab50815e2b973d422f3d715f0a4bb59f Jul 02 23:07:17.768268 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.768248 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cnq78" Jul 02 23:07:17.775873 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:17.775848 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd952dcf9_8a86_487d_9cfb_b6c592a17982.slice/crio-9c26b0c710155934ac6239d3d80d7c01b3e2dadd13a9d5dffa62b60627228809 WatchSource:0}: Error finding container 9c26b0c710155934ac6239d3d80d7c01b3e2dadd13a9d5dffa62b60627228809: Status 404 returned error can't find the container with id 9c26b0c710155934ac6239d3d80d7c01b3e2dadd13a9d5dffa62b60627228809 Jul 02 23:07:17.784394 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.784372 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ngwbb" Jul 02 23:07:17.789880 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:17.789856 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xt994" Jul 02 23:07:17.793635 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:17.793611 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3a17f85_d07c_4017_935c_fb5dfe60dec9.slice/crio-766abf2f6f8af7581cadcb1f8f3fb21e01a83c56c1ada733f20c4ca11c99a679 WatchSource:0}: Error finding container 766abf2f6f8af7581cadcb1f8f3fb21e01a83c56c1ada733f20c4ca11c99a679: Status 404 returned error can't find the container with id 766abf2f6f8af7581cadcb1f8f3fb21e01a83c56c1ada733f20c4ca11c99a679 Jul 02 23:07:17.798505 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:17.798443 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod511288f9_603d_46d3_aaf1_f1725e8f3226.slice/crio-3b443a38ba57a7b40eb6719a8a3fc1f51da61603c0dffd1612c37950f0ec3f58 WatchSource:0}: Error finding container 3b443a38ba57a7b40eb6719a8a3fc1f51da61603c0dffd1612c37950f0ec3f58: Status 404 returned error can't find the container with id 3b443a38ba57a7b40eb6719a8a3fc1f51da61603c0dffd1612c37950f0ec3f58 Jul 02 23:07:18.066682 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:18.064096 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret\") pod \"global-pull-secret-syncer-rs2g8\" (UID: \"dcb0135c-18a2-4e25-9193-16f4e2a3b833\") " pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:18.066682 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:18.065122 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs\") pod \"network-metrics-daemon-xqwp5\" (UID: \"e3f048d9-2670-488f-9a21-73f967c069b7\") " pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:18.066682 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:18.065291 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:07:18.066682 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:18.065368 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs podName:e3f048d9-2670-488f-9a21-73f967c069b7 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:19.065339309 +0000 UTC m=+3.088157402 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs") pod "network-metrics-daemon-xqwp5" (UID: "e3f048d9-2670-488f-9a21-73f967c069b7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:07:18.066682 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:18.065864 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:07:18.066682 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:18.065943 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret podName:dcb0135c-18a2-4e25-9193-16f4e2a3b833 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:19.065916114 +0000 UTC m=+3.088734210 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret") pod "global-pull-secret-syncer-rs2g8" (UID: "dcb0135c-18a2-4e25-9193-16f4e2a3b833") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:07:18.166549 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:18.165732 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzfdj\" (UniqueName: \"kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj\") pod \"network-check-target-drz87\" (UID: \"4431a02d-4597-4cb5-ad47-faaa53a10233\") " pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:18.166549 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:18.165987 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:07:18.166549 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:18.166011 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:07:18.166549 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:18.166024 2583 projected.go:196] Error preparing data for projected volume kube-api-access-kzfdj for pod openshift-network-diagnostics/network-check-target-drz87: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:07:18.166549 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:18.166088 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj podName:4431a02d-4597-4cb5-ad47-faaa53a10233 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:19.16606826 +0000 UTC m=+3.188886374 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kzfdj" (UniqueName: "kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj") pod "network-check-target-drz87" (UID: "4431a02d-4597-4cb5-ad47-faaa53a10233") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:07:18.248922 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:18.248883 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:07:18.405482 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:18.405296 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 23:02:18 +0000 UTC" deadline="2028-03-06 11:07:15.205978514 +0000 UTC" Jul 02 23:07:18.405482 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:18.405360 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14699h59m56.800623439s" Jul 02 23:07:18.556692 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:18.556629 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" event={"ID":"57aefc7d-7069-45b4-9f4d-4612bebd980d","Type":"ContainerStarted","Data":"e67d6425a744c50738c47db46592b791d79c6581b0e5b15805400003a0b1242b"} Jul 02 23:07:18.604397 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:18.604259 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xbnmr" event={"ID":"e98ff5d8-d7cf-4864-a5f1-a2eefcde889c","Type":"ContainerStarted","Data":"67921577b085b2943a0c6f337fec8d415a1ab097813e4f99a2ff198335a5de68"} Jul 02 23:07:18.628540 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:18.628070 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal" event={"ID":"e6bcbd2655d21ca4d5712d402679456b","Type":"ContainerStarted","Data":"20183d9b6bec733709ac9ccf69d85c46cad743e4025ab942d15b4e941e280659"} Jul 02 23:07:18.657414 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:18.657317 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ngwbb" event={"ID":"a3a17f85-d07c-4017-935c-fb5dfe60dec9","Type":"ContainerStarted","Data":"766abf2f6f8af7581cadcb1f8f3fb21e01a83c56c1ada733f20c4ca11c99a679"} Jul 02 23:07:18.665020 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:18.662869 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" event={"ID":"cb571463-b959-4b84-bba1-2b22b22157fd","Type":"ContainerStarted","Data":"e90bd637d2f47945a015fef0b2cf92ec3ad33b63ebc81a7fbdb14b242dec0969"} Jul 02 23:07:18.675359 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:18.675318 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b2nvt" event={"ID":"ad87676b-0e54-4723-895d-9e19afedfca7","Type":"ContainerStarted","Data":"febea4275d51784a99b206c300800709a71c35bd05692d313ae0465fbd5d9db7"} Jul 02 23:07:18.698215 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:18.698171 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-243.ec2.internal" event={"ID":"20f20beb3812fd7a1ed1315ed6bb79ac","Type":"ContainerStarted","Data":"a4c36963c1c6170482411601bd8810d51f257b86d2ca48f9edc1fc21c79367b2"} Jul 02 23:07:18.719391 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:18.719297 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xt994" event={"ID":"511288f9-603d-46d3-aaf1-f1725e8f3226","Type":"ContainerStarted","Data":"3b443a38ba57a7b40eb6719a8a3fc1f51da61603c0dffd1612c37950f0ec3f58"} Jul 02 23:07:18.730015 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:18.729890 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cnq78" event={"ID":"d952dcf9-8a86-487d-9cfb-b6c592a17982","Type":"ContainerStarted","Data":"9c26b0c710155934ac6239d3d80d7c01b3e2dadd13a9d5dffa62b60627228809"} Jul 02 23:07:18.739427 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:18.739382 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bv9xn" event={"ID":"2055d544-3273-4518-b065-e41ceba96443","Type":"ContainerStarted","Data":"ddb1d56f8dfc01f84ec1053309c678b8ab50815e2b973d422f3d715f0a4bb59f"} Jul 02 23:07:18.778800 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:18.778748 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" event={"ID":"b172564e-817c-4344-b401-44f4322766f1","Type":"ContainerStarted","Data":"d724db865f8f43cf5b4dc0b55e12b2169fc4d87e80c4b71f1e37a05ff12eddd5"} Jul 02 23:07:18.809565 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:18.809529 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:07:19.073402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:19.073358 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs\") pod \"network-metrics-daemon-xqwp5\" (UID: \"e3f048d9-2670-488f-9a21-73f967c069b7\") " pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:19.073612 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:19.073438 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret\") pod \"global-pull-secret-syncer-rs2g8\" (UID: \"dcb0135c-18a2-4e25-9193-16f4e2a3b833\") " pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:19.073612 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:19.073600 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:07:19.073733 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:19.073669 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret podName:dcb0135c-18a2-4e25-9193-16f4e2a3b833 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:21.073648762 +0000 UTC m=+5.096466893 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret") pod "global-pull-secret-syncer-rs2g8" (UID: "dcb0135c-18a2-4e25-9193-16f4e2a3b833") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:07:19.074141 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:19.074116 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:07:19.074266 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:19.074180 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs podName:e3f048d9-2670-488f-9a21-73f967c069b7 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:21.074160134 +0000 UTC m=+5.096978229 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs") pod "network-metrics-daemon-xqwp5" (UID: "e3f048d9-2670-488f-9a21-73f967c069b7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:07:19.175174 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:19.174310 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzfdj\" (UniqueName: \"kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj\") pod \"network-check-target-drz87\" (UID: \"4431a02d-4597-4cb5-ad47-faaa53a10233\") " pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:19.175174 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:19.174521 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:07:19.175174 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:19.174546 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:07:19.175174 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:19.174559 2583 projected.go:196] Error preparing data for projected volume kube-api-access-kzfdj for pod openshift-network-diagnostics/network-check-target-drz87: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:07:19.175174 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:19.174625 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj podName:4431a02d-4597-4cb5-ad47-faaa53a10233 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:21.174603984 +0000 UTC m=+5.197422099 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kzfdj" (UniqueName: "kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj") pod "network-check-target-drz87" (UID: "4431a02d-4597-4cb5-ad47-faaa53a10233") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:07:19.406246 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:19.405862 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 23:02:18 +0000 UTC" deadline="2028-01-28 21:26:03.447295747 +0000 UTC" Jul 02 23:07:19.406246 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:19.406160 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13798h18m44.041142883s" Jul 02 23:07:19.526740 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:19.526687 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:19.526935 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:19.526817 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rs2g8" podUID="dcb0135c-18a2-4e25-9193-16f4e2a3b833" Jul 02 23:07:19.527296 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:19.527273 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:19.527425 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:19.527388 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqwp5" podUID="e3f048d9-2670-488f-9a21-73f967c069b7" Jul 02 23:07:19.527493 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:19.527463 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:19.527567 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:19.527539 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drz87" podUID="4431a02d-4597-4cb5-ad47-faaa53a10233" Jul 02 23:07:19.567595 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:19.567560 2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:07:21.092232 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:21.091348 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs\") pod \"network-metrics-daemon-xqwp5\" (UID: \"e3f048d9-2670-488f-9a21-73f967c069b7\") " pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:21.092232 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:21.091427 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret\") pod \"global-pull-secret-syncer-rs2g8\" (UID: \"dcb0135c-18a2-4e25-9193-16f4e2a3b833\") " pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:21.092232 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:21.091572 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:07:21.092232 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:21.091653 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret podName:dcb0135c-18a2-4e25-9193-16f4e2a3b833 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:25.091632133 +0000 UTC m=+9.114450227 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret") pod "global-pull-secret-syncer-rs2g8" (UID: "dcb0135c-18a2-4e25-9193-16f4e2a3b833") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:07:21.092232 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:21.092113 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:07:21.092232 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:21.092165 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs podName:e3f048d9-2670-488f-9a21-73f967c069b7 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:25.092149439 +0000 UTC m=+9.114967539 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs") pod "network-metrics-daemon-xqwp5" (UID: "e3f048d9-2670-488f-9a21-73f967c069b7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:07:21.192050 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:21.191993 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzfdj\" (UniqueName: \"kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj\") pod \"network-check-target-drz87\" (UID: \"4431a02d-4597-4cb5-ad47-faaa53a10233\") " pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:21.192283 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:21.192261 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:07:21.192368 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:21.192293 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:07:21.192368 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:21.192309 2583 projected.go:196] Error preparing data for projected volume kube-api-access-kzfdj for pod openshift-network-diagnostics/network-check-target-drz87: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:07:21.192485 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:21.192378 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj podName:4431a02d-4597-4cb5-ad47-faaa53a10233 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:25.192355874 +0000 UTC m=+9.215173975 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kzfdj" (UniqueName: "kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj") pod "network-check-target-drz87" (UID: "4431a02d-4597-4cb5-ad47-faaa53a10233") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:07:21.527651 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:21.526674 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:21.527651 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:21.526834 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rs2g8" podUID="dcb0135c-18a2-4e25-9193-16f4e2a3b833" Jul 02 23:07:21.527651 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:21.527281 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:21.527651 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:21.527399 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqwp5" podUID="e3f048d9-2670-488f-9a21-73f967c069b7" Jul 02 23:07:21.527651 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:21.527451 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:21.527651 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:21.527537 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drz87" podUID="4431a02d-4597-4cb5-ad47-faaa53a10233" Jul 02 23:07:23.526922 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:23.526875 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:23.527453 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:23.527023 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:23.527453 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:23.527055 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drz87" podUID="4431a02d-4597-4cb5-ad47-faaa53a10233" Jul 02 23:07:23.527453 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:23.527145 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:23.527453 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:23.527137 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqwp5" podUID="e3f048d9-2670-488f-9a21-73f967c069b7" Jul 02 23:07:23.527453 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:23.527245 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rs2g8" podUID="dcb0135c-18a2-4e25-9193-16f4e2a3b833" Jul 02 23:07:25.127292 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:25.127250 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret\") pod \"global-pull-secret-syncer-rs2g8\" (UID: \"dcb0135c-18a2-4e25-9193-16f4e2a3b833\") " pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:25.127723 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:25.127322 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs\") pod \"network-metrics-daemon-xqwp5\" (UID: \"e3f048d9-2670-488f-9a21-73f967c069b7\") " pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:25.127723 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:25.127420 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:07:25.127723 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:25.127430 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:07:25.127723 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:25.127473 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs podName:e3f048d9-2670-488f-9a21-73f967c069b7 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:33.127456496 +0000 UTC m=+17.150274589 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs") pod "network-metrics-daemon-xqwp5" (UID: "e3f048d9-2670-488f-9a21-73f967c069b7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:07:25.127723 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:25.127496 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret podName:dcb0135c-18a2-4e25-9193-16f4e2a3b833 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:33.127481348 +0000 UTC m=+17.150299445 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret") pod "global-pull-secret-syncer-rs2g8" (UID: "dcb0135c-18a2-4e25-9193-16f4e2a3b833") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:07:25.228591 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:25.228545 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzfdj\" (UniqueName: \"kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj\") pod \"network-check-target-drz87\" (UID: \"4431a02d-4597-4cb5-ad47-faaa53a10233\") " pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:25.228774 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:25.228745 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:07:25.228774 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:25.228770 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:07:25.228774 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:25.228783 2583 projected.go:196] Error preparing data for projected volume kube-api-access-kzfdj for pod openshift-network-diagnostics/network-check-target-drz87: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:07:25.229060 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:25.228846 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj podName:4431a02d-4597-4cb5-ad47-faaa53a10233 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:33.22882666 +0000 UTC m=+17.251644791 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kzfdj" (UniqueName: "kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj") pod "network-check-target-drz87" (UID: "4431a02d-4597-4cb5-ad47-faaa53a10233") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:07:25.526541 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:25.526446 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:25.526736 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:25.526596 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rs2g8" podUID="dcb0135c-18a2-4e25-9193-16f4e2a3b833" Jul 02 23:07:25.526736 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:25.526662 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:25.526736 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:25.526726 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqwp5" podUID="e3f048d9-2670-488f-9a21-73f967c069b7" Jul 02 23:07:25.526945 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:25.526761 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:25.526945 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:25.526820 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drz87" podUID="4431a02d-4597-4cb5-ad47-faaa53a10233" Jul 02 23:07:27.526485 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:27.526439 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:27.527006 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:27.526498 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:27.527006 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:27.526498 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:27.527006 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:27.526595 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rs2g8" podUID="dcb0135c-18a2-4e25-9193-16f4e2a3b833" Jul 02 23:07:27.527006 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:27.526706 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqwp5" podUID="e3f048d9-2670-488f-9a21-73f967c069b7" Jul 02 23:07:27.527006 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:27.526773 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drz87" podUID="4431a02d-4597-4cb5-ad47-faaa53a10233" Jul 02 23:07:29.527206 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:29.527166 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:29.527206 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:29.527205 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:29.527802 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:29.527175 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:29.527802 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:29.527311 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drz87" podUID="4431a02d-4597-4cb5-ad47-faaa53a10233" Jul 02 23:07:29.527802 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:29.527426 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rs2g8" podUID="dcb0135c-18a2-4e25-9193-16f4e2a3b833" Jul 02 23:07:29.527802 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:29.527521 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqwp5" podUID="e3f048d9-2670-488f-9a21-73f967c069b7" Jul 02 23:07:31.526804 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:31.526763 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:31.527286 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:31.526763 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:31.527286 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:31.526916 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:31.527286 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:31.526914 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqwp5" podUID="e3f048d9-2670-488f-9a21-73f967c069b7" Jul 02 23:07:31.527286 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:31.527015 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rs2g8" podUID="dcb0135c-18a2-4e25-9193-16f4e2a3b833" Jul 02 23:07:31.527286 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:31.527104 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drz87" podUID="4431a02d-4597-4cb5-ad47-faaa53a10233" Jul 02 23:07:33.198146 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:33.198111 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs\") pod \"network-metrics-daemon-xqwp5\" (UID: \"e3f048d9-2670-488f-9a21-73f967c069b7\") " pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:33.198611 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:33.198160 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret\") pod \"global-pull-secret-syncer-rs2g8\" (UID: \"dcb0135c-18a2-4e25-9193-16f4e2a3b833\") " pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:33.198611 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:33.198265 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:07:33.198611 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:33.198270 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:07:33.198611 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:33.198329 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret podName:dcb0135c-18a2-4e25-9193-16f4e2a3b833 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:49.198310861 +0000 UTC m=+33.221128953 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret") pod "global-pull-secret-syncer-rs2g8" (UID: "dcb0135c-18a2-4e25-9193-16f4e2a3b833") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:07:33.198611 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:33.198357 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs podName:e3f048d9-2670-488f-9a21-73f967c069b7 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:49.198338219 +0000 UTC m=+33.221156312 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs") pod "network-metrics-daemon-xqwp5" (UID: "e3f048d9-2670-488f-9a21-73f967c069b7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:07:33.298904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:33.298853 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzfdj\" (UniqueName: \"kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj\") pod \"network-check-target-drz87\" (UID: \"4431a02d-4597-4cb5-ad47-faaa53a10233\") " pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:33.299083 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:33.299022 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:07:33.299083 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:33.299047 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:07:33.299083 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:33.299060 2583 projected.go:196] Error preparing data for projected volume kube-api-access-kzfdj for pod openshift-network-diagnostics/network-check-target-drz87: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:07:33.299201 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:33.299113 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj podName:4431a02d-4597-4cb5-ad47-faaa53a10233 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:49.299098571 +0000 UTC m=+33.321916662 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kzfdj" (UniqueName: "kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj") pod "network-check-target-drz87" (UID: "4431a02d-4597-4cb5-ad47-faaa53a10233") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:07:33.527053 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:33.526971 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:33.527053 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:33.527009 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:33.527286 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:33.527108 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqwp5" podUID="e3f048d9-2670-488f-9a21-73f967c069b7" Jul 02 23:07:33.527286 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:33.527148 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:33.527286 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:33.527226 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drz87" podUID="4431a02d-4597-4cb5-ad47-faaa53a10233" Jul 02 23:07:33.527423 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:33.527308 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rs2g8" podUID="dcb0135c-18a2-4e25-9193-16f4e2a3b833" Jul 02 23:07:35.526560 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:35.526523 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:35.527069 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:35.526633 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:35.527069 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:35.526637 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drz87" podUID="4431a02d-4597-4cb5-ad47-faaa53a10233" Jul 02 23:07:35.527069 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:35.526738 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rs2g8" podUID="dcb0135c-18a2-4e25-9193-16f4e2a3b833" Jul 02 23:07:35.527069 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:35.526638 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:35.527069 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:35.526826 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqwp5" podUID="e3f048d9-2670-488f-9a21-73f967c069b7" Jul 02 23:07:37.526822 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:37.526782 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:37.527305 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:37.526781 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:37.527305 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:37.526935 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqwp5" podUID="e3f048d9-2670-488f-9a21-73f967c069b7" Jul 02 23:07:37.527305 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:37.526802 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:37.527305 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:37.526985 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rs2g8" podUID="dcb0135c-18a2-4e25-9193-16f4e2a3b833" Jul 02 23:07:37.527305 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:37.527094 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drz87" podUID="4431a02d-4597-4cb5-ad47-faaa53a10233" Jul 02 23:07:39.526425 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.526231 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:39.527093 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.526274 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:39.527093 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:39.526509 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drz87" podUID="4431a02d-4597-4cb5-ad47-faaa53a10233" Jul 02 23:07:39.527093 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:39.526618 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rs2g8" podUID="dcb0135c-18a2-4e25-9193-16f4e2a3b833" Jul 02 23:07:39.527093 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.526299 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:39.527093 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:39.526722 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqwp5" podUID="e3f048d9-2670-488f-9a21-73f967c069b7" Jul 02 23:07:39.839736 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.839533 2583 generic.go:356] "Generic (PLEG): container finished" podID="e6bcbd2655d21ca4d5712d402679456b" containerID="c33e6cb417b10c6c9840640c33070540bc21cdc50d6787070d4549fbb3f6e1fb" exitCode=0 Jul 02 23:07:39.839958 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.839617 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal" event={"ID":"e6bcbd2655d21ca4d5712d402679456b","Type":"ContainerDied","Data":"c33e6cb417b10c6c9840640c33070540bc21cdc50d6787070d4549fbb3f6e1fb"} Jul 02 23:07:39.841447 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.841421 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ngwbb" event={"ID":"a3a17f85-d07c-4017-935c-fb5dfe60dec9","Type":"ContainerStarted","Data":"f90298a82942f64d257cb18dd916999110181cdf89035d6f2670984ad8d14e5e"} Jul 02 23:07:39.847510 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.847483 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" event={"ID":"cb571463-b959-4b84-bba1-2b22b22157fd","Type":"ContainerStarted","Data":"e0cf274affc2f06b344e4d059935dcf36d24d9dee60db13e1b15bf5fa6b946c8"} Jul 02 23:07:39.847646 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.847588 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" event={"ID":"cb571463-b959-4b84-bba1-2b22b22157fd","Type":"ContainerStarted","Data":"643add20a560a864bda1552d5cc2e8e5622cc104276bd483442c526502266111"} Jul 02 23:07:39.847646 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.847638 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" event={"ID":"cb571463-b959-4b84-bba1-2b22b22157fd","Type":"ContainerStarted","Data":"8d79b64dc86ad2111a6fb539fcce7306fca3f950ac6c202e06ed7cadc9ed0210"} Jul 02 23:07:39.847757 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.847684 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" event={"ID":"cb571463-b959-4b84-bba1-2b22b22157fd","Type":"ContainerStarted","Data":"9fa2def75a7a0e4c916db9ae4d305d9b0dc2bce9b7dc0ecb44d81a4684039cf8"} Jul 02 23:07:39.847757 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.847731 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" event={"ID":"cb571463-b959-4b84-bba1-2b22b22157fd","Type":"ContainerStarted","Data":"de9c041a3538be77099b05e4724227d68cb240b220360663863a18435babde51"} Jul 02 23:07:39.847852 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.847778 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" event={"ID":"cb571463-b959-4b84-bba1-2b22b22157fd","Type":"ContainerStarted","Data":"f3a08c55982fbffd76dcc58a1ecc6b366ac7b301b2b2c65afdf0b1d2e6a57b3a"} Jul 02 23:07:39.848955 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.848931 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b2nvt" event={"ID":"ad87676b-0e54-4723-895d-9e19afedfca7","Type":"ContainerStarted","Data":"85818f87b47ee7282211a7a3e71548f8fe1590b52ae8cd562acca340e72fbd17"} Jul 02 23:07:39.850277 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.850252 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-243.ec2.internal" event={"ID":"20f20beb3812fd7a1ed1315ed6bb79ac","Type":"ContainerStarted","Data":"f538b23f6a8e3013a7f565f9157e3928b159725518e4d24f09d829d414ab8e42"} Jul 02 23:07:39.851626 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.851598 2583 generic.go:356] "Generic (PLEG): container finished" podID="511288f9-603d-46d3-aaf1-f1725e8f3226" containerID="55140fb2d7ba20865e63f964022b6ac24aef5faae1ad4ff9721ed3be5f668f09" exitCode=0 Jul 02 23:07:39.851721 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.851664 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xt994" event={"ID":"511288f9-603d-46d3-aaf1-f1725e8f3226","Type":"ContainerDied","Data":"55140fb2d7ba20865e63f964022b6ac24aef5faae1ad4ff9721ed3be5f668f09"} Jul 02 23:07:39.853263 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.853234 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cnq78" event={"ID":"d952dcf9-8a86-487d-9cfb-b6c592a17982","Type":"ContainerStarted","Data":"ad35b4b93193ff76b877febf62b2d6f8b58f4a940808cd603d8ebcb8cad07edb"} Jul 02 23:07:39.854640 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.854617 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bv9xn" event={"ID":"2055d544-3273-4518-b065-e41ceba96443","Type":"ContainerStarted","Data":"3b2ad3d8821c5c83e39f6194cbc8e6750fd5a73f7cd182127a935ea686b4c7a9"} Jul 02 23:07:39.856021 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.855987 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" event={"ID":"b172564e-817c-4344-b401-44f4322766f1","Type":"ContainerStarted","Data":"ce8322c6cf924cf8b8a2719a3fd1e33aef0d363d438f52fbb3df955f21fcc590"} Jul 02 23:07:39.857290 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.857271 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" event={"ID":"57aefc7d-7069-45b4-9f4d-4612bebd980d","Type":"ContainerStarted","Data":"548767b985853e305a94c388618100b8221431db0e723cb3a58cbde85629ae9b"} Jul 02 23:07:39.892460 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.892397 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-b2nvt" podStartSLOduration=1.730354247 podStartE2EDuration="22.892375992s" totalImagesPullingTime="21.162021745s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:07:39.874105335 +0000 UTC m=+23.896923449" watchObservedRunningTime="2026-07-02 23:07:39.892375992 +0000 UTC m=+23.915194106" Jul 02 23:07:39.904023 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.903960 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-243.ec2.internal" podStartSLOduration=21.903939896 podStartE2EDuration="21.903939896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:07:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:07:39.90334729 +0000 UTC m=+23.926165405" watchObservedRunningTime="2026-07-02 23:07:39.903939896 +0000 UTC m=+23.926758009" Jul 02 23:07:39.919646 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.919598 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ngwbb" podStartSLOduration=1.86866794 podStartE2EDuration="22.919582419s" totalImagesPullingTime="21.050914479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:07:39.918821136 +0000 UTC m=+23.941639249" watchObservedRunningTime="2026-07-02 23:07:39.919582419 +0000 UTC m=+23.942400533" Jul 02 23:07:39.935172 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.935126 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cnq78" podStartSLOduration=1.811421131 podStartE2EDuration="22.935111229s" totalImagesPullingTime="21.123690098s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:07:39.934943752 +0000 UTC m=+23.957761865" watchObservedRunningTime="2026-07-02 23:07:39.935111229 +0000 UTC m=+23.957929341" Jul 02 23:07:39.950423 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.950378 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bv9xn" podStartSLOduration=1.856490259 podStartE2EDuration="22.950362265s" totalImagesPullingTime="21.093872006s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:07:39.949940911 +0000 UTC m=+23.972759044" watchObservedRunningTime="2026-07-02 23:07:39.950362265 +0000 UTC m=+23.973180375" Jul 02 23:07:39.967737 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:39.967695 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hd6kw" podStartSLOduration=1.868147531 podStartE2EDuration="22.967680007s" totalImagesPullingTime="21.099532476s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:07:39.96746843 +0000 UTC m=+23.990286545" watchObservedRunningTime="2026-07-02 23:07:39.967680007 +0000 UTC m=+23.990498098" Jul 02 23:07:40.085524 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:40.085470 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ngwbb_a3a17f85-d07c-4017-935c-fb5dfe60dec9/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:40.165728 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:40.165621 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ngwbb_a3a17f85-d07c-4017-935c-fb5dfe60dec9/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:40.861087 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:40.860982 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xbnmr" event={"ID":"e98ff5d8-d7cf-4864-a5f1-a2eefcde889c","Type":"ContainerStarted","Data":"b08f7db314a00293d26cf9026d775218561e6cf0b9748c98e175ee7aa8954c2e"} Jul 02 23:07:40.863102 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:40.863069 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal" event={"ID":"e6bcbd2655d21ca4d5712d402679456b","Type":"ContainerStarted","Data":"1a0d35d999c532b0eaf88f598578266cd2cf5de406df8c5b08c7a9680b72200a"} Jul 02 23:07:40.874674 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:40.874626 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xbnmr" podStartSLOduration=2.707868956 podStartE2EDuration="23.874613489s" totalImagesPullingTime="21.166744533s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:07:40.874240323 +0000 UTC m=+24.897058449" watchObservedRunningTime="2026-07-02 23:07:40.874613489 +0000 UTC m=+24.897431601" Jul 02 23:07:40.889320 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:40.889279 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-243.ec2.internal" podStartSLOduration=22.889266334 podStartE2EDuration="22.889266334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:07:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:07:40.888618398 +0000 UTC m=+24.911436512" watchObservedRunningTime="2026-07-02 23:07:40.889266334 +0000 UTC m=+24.912084446" Jul 02 23:07:41.105387 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:41.105366 2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 23:07:41.436422 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:41.436343 2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T23:07:41.105386524Z","UUID":"eb3ae345-6850-4c46-9129-5a40add2903b","Handler":null,"Name":"","Endpoint":""} Jul 02 23:07:41.437659 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:41.437636 2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 23:07:41.437778 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:41.437674 2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 23:07:41.527198 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:41.527162 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:41.527406 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:41.527292 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:41.527406 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:41.527304 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rs2g8" podUID="dcb0135c-18a2-4e25-9193-16f4e2a3b833" Jul 02 23:07:41.527406 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:41.527326 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:41.527570 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:41.527424 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drz87" podUID="4431a02d-4597-4cb5-ad47-faaa53a10233" Jul 02 23:07:41.527570 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:41.527523 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqwp5" podUID="e3f048d9-2670-488f-9a21-73f967c069b7" Jul 02 23:07:41.867533 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:41.867490 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" event={"ID":"57aefc7d-7069-45b4-9f4d-4612bebd980d","Type":"ContainerStarted","Data":"a22ded4ea2d0fbac85a0af4df9240aa23ba4690806de483aae558d7920286067"} Jul 02 23:07:42.871715 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:42.871680 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" event={"ID":"57aefc7d-7069-45b4-9f4d-4612bebd980d","Type":"ContainerStarted","Data":"5d7f6d27471b078ac73e727ce4a610ac381df73e9cd0c627555699ce81baa6d5"} Jul 02 23:07:42.874852 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:42.874824 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" event={"ID":"cb571463-b959-4b84-bba1-2b22b22157fd","Type":"ContainerStarted","Data":"482715a98ffdab668577bbb7a30975b4af35c0987e6adbcc0f760968a723a35b"} Jul 02 23:07:42.888304 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:42.888267 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdgmq" podStartSLOduration=2.065583408 podStartE2EDuration="25.888253737s" totalImagesPullingTime="23.822670329s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:07:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:07:42.886851773 +0000 UTC m=+26.909669886" watchObservedRunningTime="2026-07-02 23:07:42.888253737 +0000 UTC m=+26.911071877" Jul 02 23:07:43.342662 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:43.342401 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ngwbb" Jul 02 23:07:43.343043 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:43.343026 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ngwbb" Jul 02 23:07:43.526764 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:43.526725 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:43.526764 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:43.526746 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:43.526764 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:43.526725 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:43.527029 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:43.526837 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drz87" podUID="4431a02d-4597-4cb5-ad47-faaa53a10233" Jul 02 23:07:43.527029 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:43.526972 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqwp5" podUID="e3f048d9-2670-488f-9a21-73f967c069b7" Jul 02 23:07:43.527095 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:43.527039 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rs2g8" podUID="dcb0135c-18a2-4e25-9193-16f4e2a3b833" Jul 02 23:07:43.877751 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:43.877721 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ngwbb" Jul 02 23:07:43.878490 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:43.878467 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ngwbb" Jul 02 23:07:45.526972 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:45.526923 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:45.527537 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:45.526976 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:45.527537 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:45.527056 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rs2g8" podUID="dcb0135c-18a2-4e25-9193-16f4e2a3b833" Jul 02 23:07:45.527537 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:45.527137 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:45.527537 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:45.527408 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drz87" podUID="4431a02d-4597-4cb5-ad47-faaa53a10233" Jul 02 23:07:45.527670 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:45.527550 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqwp5" podUID="e3f048d9-2670-488f-9a21-73f967c069b7" Jul 02 23:07:45.885663 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:45.885365 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" event={"ID":"cb571463-b959-4b84-bba1-2b22b22157fd","Type":"ContainerStarted","Data":"67654985d529afcfba1e812156d2a68539d9cf24089d779413ea0113b15cb8df"} Jul 02 23:07:45.885812 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:45.885673 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:45.887210 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:45.887182 2583 generic.go:356] "Generic (PLEG): container finished" podID="511288f9-603d-46d3-aaf1-f1725e8f3226" containerID="bcdef4a43bfb02a30b348964770a48c0d0641d78aedd13f31e9cad42b7ea8de3" exitCode=0 Jul 02 23:07:45.887346 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:45.887252 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xt994" event={"ID":"511288f9-603d-46d3-aaf1-f1725e8f3226","Type":"ContainerDied","Data":"bcdef4a43bfb02a30b348964770a48c0d0641d78aedd13f31e9cad42b7ea8de3"} Jul 02 23:07:45.902354 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:45.902324 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:45.910573 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:45.910530 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" podStartSLOduration=7.715595763 podStartE2EDuration="28.910516734s" totalImagesPullingTime="21.194920971s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:07:45.909867574 +0000 UTC m=+29.932685685" watchObservedRunningTime="2026-07-02 23:07:45.910516734 +0000 UTC m=+29.933334895" Jul 02 23:07:46.889809 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:46.889772 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:46.890276 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:46.889875 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:46.904703 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:46.904677 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:07:47.526460 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:47.526422 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:47.526460 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:47.526451 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:47.526699 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:47.526468 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:47.526699 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:47.526562 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rs2g8" podUID="dcb0135c-18a2-4e25-9193-16f4e2a3b833" Jul 02 23:07:47.526699 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:47.526683 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqwp5" podUID="e3f048d9-2670-488f-9a21-73f967c069b7" Jul 02 23:07:47.526839 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:47.526758 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drz87" podUID="4431a02d-4597-4cb5-ad47-faaa53a10233" Jul 02 23:07:47.629398 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:47.629364 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xqwp5"] Jul 02 23:07:47.632222 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:47.632191 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rs2g8"] Jul 02 23:07:47.632784 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:47.632762 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-drz87"] Jul 02 23:07:47.893812 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:47.893779 2583 generic.go:356] "Generic (PLEG): container finished" podID="511288f9-603d-46d3-aaf1-f1725e8f3226" containerID="3a81b7fb53f6120e51a73545a7505048543e47db53c47883c0a4b4abc2e7e77b" exitCode=0 Jul 02 23:07:47.894264 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:47.893822 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xt994" event={"ID":"511288f9-603d-46d3-aaf1-f1725e8f3226","Type":"ContainerDied","Data":"3a81b7fb53f6120e51a73545a7505048543e47db53c47883c0a4b4abc2e7e77b"} Jul 02 23:07:47.894264 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:47.893922 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:47.894264 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:47.894031 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqwp5" podUID="e3f048d9-2670-488f-9a21-73f967c069b7" Jul 02 23:07:47.894427 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:47.894372 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:47.894546 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:47.894487 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rs2g8" podUID="dcb0135c-18a2-4e25-9193-16f4e2a3b833" Jul 02 23:07:47.894546 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:47.894538 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:47.894658 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:47.894618 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drz87" podUID="4431a02d-4597-4cb5-ad47-faaa53a10233" Jul 02 23:07:49.212122 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:49.212085 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs\") pod \"network-metrics-daemon-xqwp5\" (UID: \"e3f048d9-2670-488f-9a21-73f967c069b7\") " pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:49.212545 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:49.212136 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret\") pod \"global-pull-secret-syncer-rs2g8\" (UID: \"dcb0135c-18a2-4e25-9193-16f4e2a3b833\") " pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:49.212545 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:49.212238 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:07:49.212545 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:49.212246 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:07:49.212545 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:49.212295 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret podName:dcb0135c-18a2-4e25-9193-16f4e2a3b833 nodeName:}" failed. No retries permitted until 2026-07-02 23:08:21.21228034 +0000 UTC m=+65.235098431 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret") pod "global-pull-secret-syncer-rs2g8" (UID: "dcb0135c-18a2-4e25-9193-16f4e2a3b833") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:07:49.212545 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:49.212319 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs podName:e3f048d9-2670-488f-9a21-73f967c069b7 nodeName:}" failed. No retries permitted until 2026-07-02 23:08:21.212300696 +0000 UTC m=+65.235118791 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs") pod "network-metrics-daemon-xqwp5" (UID: "e3f048d9-2670-488f-9a21-73f967c069b7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:07:49.312851 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:49.312813 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzfdj\" (UniqueName: \"kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj\") pod \"network-check-target-drz87\" (UID: \"4431a02d-4597-4cb5-ad47-faaa53a10233\") " pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:49.313041 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:49.312977 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:07:49.313041 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:49.312994 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:07:49.313041 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:49.313002 2583 projected.go:196] Error preparing data for projected volume kube-api-access-kzfdj for pod openshift-network-diagnostics/network-check-target-drz87: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:07:49.313154 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:49.313054 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj podName:4431a02d-4597-4cb5-ad47-faaa53a10233 nodeName:}" failed. No retries permitted until 2026-07-02 23:08:21.313039329 +0000 UTC m=+65.335857420 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kzfdj" (UniqueName: "kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj") pod "network-check-target-drz87" (UID: "4431a02d-4597-4cb5-ad47-faaa53a10233") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:07:49.526800 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:49.526768 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:49.527021 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:49.526868 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:49.527021 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:49.526874 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rs2g8" podUID="dcb0135c-18a2-4e25-9193-16f4e2a3b833" Jul 02 23:07:49.527160 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:49.527014 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drz87" podUID="4431a02d-4597-4cb5-ad47-faaa53a10233" Jul 02 23:07:49.527160 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:49.527047 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:49.527160 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:49.527143 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqwp5" podUID="e3f048d9-2670-488f-9a21-73f967c069b7" Jul 02 23:07:49.900222 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:49.900189 2583 generic.go:356] "Generic (PLEG): container finished" podID="511288f9-603d-46d3-aaf1-f1725e8f3226" containerID="6cb2ad45fcf7e34e7e1cffab0b668f6c04b5b90741956e944a8b7200c59480ea" exitCode=0 Jul 02 23:07:49.900402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:49.900257 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xt994" event={"ID":"511288f9-603d-46d3-aaf1-f1725e8f3226","Type":"ContainerDied","Data":"6cb2ad45fcf7e34e7e1cffab0b668f6c04b5b90741956e944a8b7200c59480ea"} Jul 02 23:07:51.341735 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:51.341704 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b2nvt_ad87676b-0e54-4723-895d-9e19afedfca7/dns-node-resolver/0.log" Jul 02 23:07:51.527044 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:51.527008 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:51.527044 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:51.527050 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:51.527228 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:51.527011 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:51.527228 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:51.527112 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drz87" podUID="4431a02d-4597-4cb5-ad47-faaa53a10233" Jul 02 23:07:51.527228 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:51.527183 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqwp5" podUID="e3f048d9-2670-488f-9a21-73f967c069b7" Jul 02 23:07:51.527324 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:51.527266 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rs2g8" podUID="dcb0135c-18a2-4e25-9193-16f4e2a3b833" Jul 02 23:07:52.350076 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.350043 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bv9xn_2055d544-3273-4518-b065-e41ceba96443/node-ca/0.log" Jul 02 23:07:52.722133 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.722045 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-243.ec2.internal" event="NodeReady" Jul 02 23:07:52.722300 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.722212 2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 23:07:52.766740 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.766707 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vblxw"] Jul 02 23:07:52.769407 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.769381 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bqx8m"] Jul 02 23:07:52.770174 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.770158 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vblxw" Jul 02 23:07:52.771152 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.771132 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:52.772590 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.772572 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 23:07:52.773008 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.772986 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 23:07:52.773197 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.773178 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rwcd4\"" Jul 02 23:07:52.773300 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.773269 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 23:07:52.773300 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.773283 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 23:07:52.773395 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.773336 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xpm5t\"" Jul 02 23:07:52.773688 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.773668 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 23:07:52.773770 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.773705 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 23:07:52.773770 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.773668 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 23:07:52.780222 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.780099 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vblxw"] Jul 02 23:07:52.783606 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.783543 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bqx8m"] Jul 02 23:07:52.790898 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.790854 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2vvfs"] Jul 02 23:07:52.793210 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.792769 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2vvfs" Jul 02 23:07:52.794956 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.794937 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 23:07:52.795044 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.794943 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dl5xp\"" Jul 02 23:07:52.795044 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.794943 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 23:07:52.808528 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.808492 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2vvfs"] Jul 02 23:07:52.940244 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.940209 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6jh4\" (UniqueName: \"kubernetes.io/projected/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-kube-api-access-n6jh4\") pod \"dns-default-2vvfs\" (UID: \"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79\") " pod="openshift-dns/dns-default-2vvfs" Jul 02 23:07:52.940244 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.940250 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/952de78d-e0fb-4c00-88d6-ca117114fd67-cert\") pod \"ingress-canary-vblxw\" (UID: \"952de78d-e0fb-4c00-88d6-ca117114fd67\") " pod="openshift-ingress-canary/ingress-canary-vblxw" Jul 02 23:07:52.940478 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.940304 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:52.940478 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.940325 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzx4s\" (UniqueName: \"kubernetes.io/projected/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-kube-api-access-qzx4s\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:52.940478 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.940356 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-crio-socket\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:52.940478 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.940397 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-config-volume\") pod \"dns-default-2vvfs\" (UID: \"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79\") " pod="openshift-dns/dns-default-2vvfs" Jul 02 23:07:52.940478 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.940419 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:52.940478 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.940438 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-metrics-tls\") pod \"dns-default-2vvfs\" (UID: \"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79\") " pod="openshift-dns/dns-default-2vvfs" Jul 02 23:07:52.940699 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.940495 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-tmp-dir\") pod \"dns-default-2vvfs\" (UID: \"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79\") " pod="openshift-dns/dns-default-2vvfs" Jul 02 23:07:52.940699 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.940521 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-data-volume\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:52.940699 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:52.940593 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmn2b\" (UniqueName: \"kubernetes.io/projected/952de78d-e0fb-4c00-88d6-ca117114fd67-kube-api-access-fmn2b\") pod \"ingress-canary-vblxw\" (UID: \"952de78d-e0fb-4c00-88d6-ca117114fd67\") " pod="openshift-ingress-canary/ingress-canary-vblxw" Jul 02 23:07:53.041556 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.041476 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-config-volume\") pod \"dns-default-2vvfs\" (UID: \"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79\") " pod="openshift-dns/dns-default-2vvfs" Jul 02 23:07:53.041556 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.041520 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:53.041556 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.041543 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-metrics-tls\") pod \"dns-default-2vvfs\" (UID: \"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79\") " pod="openshift-dns/dns-default-2vvfs" Jul 02 23:07:53.041788 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:53.041656 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:07:53.041788 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.041676 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-tmp-dir\") pod \"dns-default-2vvfs\" (UID: \"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79\") " pod="openshift-dns/dns-default-2vvfs" Jul 02 23:07:53.041788 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:53.041716 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-metrics-tls podName:fb8c01ab-cb1f-403b-aef4-c4a7143b7c79 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:53.541697821 +0000 UTC m=+37.564515924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-metrics-tls") pod "dns-default-2vvfs" (UID: "fb8c01ab-cb1f-403b-aef4-c4a7143b7c79") : secret "dns-default-metrics-tls" not found Jul 02 23:07:53.041788 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.041733 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-data-volume\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:53.041788 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:53.041656 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:07:53.042055 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.041792 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmn2b\" (UniqueName: \"kubernetes.io/projected/952de78d-e0fb-4c00-88d6-ca117114fd67-kube-api-access-fmn2b\") pod \"ingress-canary-vblxw\" (UID: \"952de78d-e0fb-4c00-88d6-ca117114fd67\") " pod="openshift-ingress-canary/ingress-canary-vblxw" Jul 02 23:07:53.042055 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:53.041828 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-insights-runtime-extractor-tls podName:57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:53.541817896 +0000 UTC m=+37.564635986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqx8m" (UID: "57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02") : secret "insights-runtime-extractor-tls" not found Jul 02 23:07:53.042055 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.041851 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6jh4\" (UniqueName: \"kubernetes.io/projected/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-kube-api-access-n6jh4\") pod \"dns-default-2vvfs\" (UID: \"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79\") " pod="openshift-dns/dns-default-2vvfs" Jul 02 23:07:53.042055 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.041876 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/952de78d-e0fb-4c00-88d6-ca117114fd67-cert\") pod \"ingress-canary-vblxw\" (UID: \"952de78d-e0fb-4c00-88d6-ca117114fd67\") " pod="openshift-ingress-canary/ingress-canary-vblxw" Jul 02 23:07:53.042055 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.041930 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:53.042055 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.041953 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzx4s\" (UniqueName: \"kubernetes.io/projected/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-kube-api-access-qzx4s\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:53.042055 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.041985 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-crio-socket\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:53.042055 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.042022 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-tmp-dir\") pod \"dns-default-2vvfs\" (UID: \"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79\") " pod="openshift-dns/dns-default-2vvfs" Jul 02 23:07:53.042435 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.042137 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-config-volume\") pod \"dns-default-2vvfs\" (UID: \"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79\") " pod="openshift-dns/dns-default-2vvfs" Jul 02 23:07:53.042435 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.042161 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-data-volume\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:53.042435 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:53.042211 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:07:53.042435 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.042214 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-crio-socket\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:53.042435 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:53.042272 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-kube-rbac-proxy-cm podName:57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:53.542257293 +0000 UTC m=+37.565075388 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bqx8m" (UID: "57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02") : configmap "kube-rbac-proxy" not found Jul 02 23:07:53.046503 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.046483 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/952de78d-e0fb-4c00-88d6-ca117114fd67-cert\") pod \"ingress-canary-vblxw\" (UID: \"952de78d-e0fb-4c00-88d6-ca117114fd67\") " pod="openshift-ingress-canary/ingress-canary-vblxw" Jul 02 23:07:53.053443 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.053133 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmn2b\" (UniqueName: \"kubernetes.io/projected/952de78d-e0fb-4c00-88d6-ca117114fd67-kube-api-access-fmn2b\") pod \"ingress-canary-vblxw\" (UID: \"952de78d-e0fb-4c00-88d6-ca117114fd67\") " pod="openshift-ingress-canary/ingress-canary-vblxw" Jul 02 23:07:53.053443 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.053202 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6jh4\" (UniqueName: \"kubernetes.io/projected/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-kube-api-access-n6jh4\") pod \"dns-default-2vvfs\" (UID: \"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79\") " pod="openshift-dns/dns-default-2vvfs" Jul 02 23:07:53.053443 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.053407 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzx4s\" (UniqueName: \"kubernetes.io/projected/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-kube-api-access-qzx4s\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:53.083987 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.083945 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vblxw" Jul 02 23:07:53.147857 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.147823 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-tx9tm"] Jul 02 23:07:53.149878 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.149857 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-tx9tm" Jul 02 23:07:53.153350 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.153324 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 02 23:07:53.153350 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.153357 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 02 23:07:53.153526 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.153423 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-9lz5h\"" Jul 02 23:07:53.153526 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.153478 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 02 23:07:53.153601 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.153555 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 02 23:07:53.157328 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.157304 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-tx9tm"] Jul 02 23:07:53.244328 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.244291 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt795\" (UniqueName: \"kubernetes.io/projected/62b1e0d4-ca21-4def-85ea-e2f9b674834b-kube-api-access-pt795\") pod \"service-ca-758f599c5-tx9tm\" (UID: \"62b1e0d4-ca21-4def-85ea-e2f9b674834b\") " pod="openshift-service-ca/service-ca-758f599c5-tx9tm" Jul 02 23:07:53.244510 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.244346 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/62b1e0d4-ca21-4def-85ea-e2f9b674834b-signing-cabundle\") pod \"service-ca-758f599c5-tx9tm\" (UID: \"62b1e0d4-ca21-4def-85ea-e2f9b674834b\") " pod="openshift-service-ca/service-ca-758f599c5-tx9tm" Jul 02 23:07:53.244510 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.244449 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/62b1e0d4-ca21-4def-85ea-e2f9b674834b-signing-key\") pod \"service-ca-758f599c5-tx9tm\" (UID: \"62b1e0d4-ca21-4def-85ea-e2f9b674834b\") " pod="openshift-service-ca/service-ca-758f599c5-tx9tm" Jul 02 23:07:53.345032 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.345001 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/62b1e0d4-ca21-4def-85ea-e2f9b674834b-signing-key\") pod \"service-ca-758f599c5-tx9tm\" (UID: \"62b1e0d4-ca21-4def-85ea-e2f9b674834b\") " pod="openshift-service-ca/service-ca-758f599c5-tx9tm" Jul 02 23:07:53.345204 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.345046 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt795\" (UniqueName: \"kubernetes.io/projected/62b1e0d4-ca21-4def-85ea-e2f9b674834b-kube-api-access-pt795\") pod \"service-ca-758f599c5-tx9tm\" (UID: \"62b1e0d4-ca21-4def-85ea-e2f9b674834b\") " pod="openshift-service-ca/service-ca-758f599c5-tx9tm" Jul 02 23:07:53.345204 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.345083 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/62b1e0d4-ca21-4def-85ea-e2f9b674834b-signing-cabundle\") pod \"service-ca-758f599c5-tx9tm\" (UID: \"62b1e0d4-ca21-4def-85ea-e2f9b674834b\") " pod="openshift-service-ca/service-ca-758f599c5-tx9tm" Jul 02 23:07:53.345818 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.345759 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/62b1e0d4-ca21-4def-85ea-e2f9b674834b-signing-cabundle\") pod \"service-ca-758f599c5-tx9tm\" (UID: \"62b1e0d4-ca21-4def-85ea-e2f9b674834b\") " pod="openshift-service-ca/service-ca-758f599c5-tx9tm" Jul 02 23:07:53.347878 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.347859 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/62b1e0d4-ca21-4def-85ea-e2f9b674834b-signing-key\") pod \"service-ca-758f599c5-tx9tm\" (UID: \"62b1e0d4-ca21-4def-85ea-e2f9b674834b\") " pod="openshift-service-ca/service-ca-758f599c5-tx9tm" Jul 02 23:07:53.353394 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.353371 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt795\" (UniqueName: \"kubernetes.io/projected/62b1e0d4-ca21-4def-85ea-e2f9b674834b-kube-api-access-pt795\") pod \"service-ca-758f599c5-tx9tm\" (UID: \"62b1e0d4-ca21-4def-85ea-e2f9b674834b\") " pod="openshift-service-ca/service-ca-758f599c5-tx9tm" Jul 02 23:07:53.460703 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.460661 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-tx9tm" Jul 02 23:07:53.527069 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.527028 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:07:53.527261 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.527079 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:07:53.527261 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.527146 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:07:53.529679 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.529657 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 23:07:53.529814 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.529757 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jkmkq\"" Jul 02 23:07:53.529814 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.529780 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 23:07:53.529901 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.529659 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 23:07:53.530111 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.530087 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 23:07:53.530210 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.530198 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dptv4\"" Jul 02 23:07:53.546493 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.546472 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:53.546649 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.546516 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:53.546649 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:53.546538 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-metrics-tls\") pod \"dns-default-2vvfs\" (UID: \"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79\") " pod="openshift-dns/dns-default-2vvfs" Jul 02 23:07:53.546649 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:53.546606 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:07:53.546649 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:53.546615 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:07:53.546649 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:53.546639 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:07:53.546859 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:53.546659 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-insights-runtime-extractor-tls podName:57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:54.546645838 +0000 UTC m=+38.569463929 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqx8m" (UID: "57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02") : secret "insights-runtime-extractor-tls" not found Jul 02 23:07:53.546859 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:53.546682 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-kube-rbac-proxy-cm podName:57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:54.546664074 +0000 UTC m=+38.569482187 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bqx8m" (UID: "57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02") : configmap "kube-rbac-proxy" not found Jul 02 23:07:53.546859 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:53.546702 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-metrics-tls podName:fb8c01ab-cb1f-403b-aef4-c4a7143b7c79 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:54.546691973 +0000 UTC m=+38.569510069 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-metrics-tls") pod "dns-default-2vvfs" (UID: "fb8c01ab-cb1f-403b-aef4-c4a7143b7c79") : secret "dns-default-metrics-tls" not found Jul 02 23:07:54.490685 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:54.490531 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-tx9tm"] Jul 02 23:07:54.497535 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:54.497509 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62b1e0d4_ca21_4def_85ea_e2f9b674834b.slice/crio-3bc3b69176ecb52f789848e6a7e49f04e8c4521dc630aac96f1b827d8e1d728d WatchSource:0}: Error finding container 3bc3b69176ecb52f789848e6a7e49f04e8c4521dc630aac96f1b827d8e1d728d: Status 404 returned error can't find the container with id 3bc3b69176ecb52f789848e6a7e49f04e8c4521dc630aac96f1b827d8e1d728d Jul 02 23:07:54.499835 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:54.499677 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vblxw"] Jul 02 23:07:54.503480 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:07:54.503460 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod952de78d_e0fb_4c00_88d6_ca117114fd67.slice/crio-06c12150791b9a47df8a6cd2c2ffef0ed2da399b2bf41f2da5a4af2180a2c868 WatchSource:0}: Error finding container 06c12150791b9a47df8a6cd2c2ffef0ed2da399b2bf41f2da5a4af2180a2c868: Status 404 returned error can't find the container with id 06c12150791b9a47df8a6cd2c2ffef0ed2da399b2bf41f2da5a4af2180a2c868 Jul 02 23:07:54.555075 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:54.555042 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:54.555075 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:54.555082 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:54.555341 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:54.555103 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-metrics-tls\") pod \"dns-default-2vvfs\" (UID: \"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79\") " pod="openshift-dns/dns-default-2vvfs" Jul 02 23:07:54.555341 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:54.555201 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:07:54.555341 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:54.555209 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:07:54.555341 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:54.555258 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-metrics-tls podName:fb8c01ab-cb1f-403b-aef4-c4a7143b7c79 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:56.555244932 +0000 UTC m=+40.578063022 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-metrics-tls") pod "dns-default-2vvfs" (UID: "fb8c01ab-cb1f-403b-aef4-c4a7143b7c79") : secret "dns-default-metrics-tls" not found Jul 02 23:07:54.555341 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:54.555273 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-insights-runtime-extractor-tls podName:57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02 nodeName:}" failed. No retries permitted until 2026-07-02 23:07:56.555265864 +0000 UTC m=+40.578083955 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqx8m" (UID: "57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02") : secret "insights-runtime-extractor-tls" not found Jul 02 23:07:54.555603 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:54.555580 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:54.914550 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:54.914456 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vblxw" event={"ID":"952de78d-e0fb-4c00-88d6-ca117114fd67","Type":"ContainerStarted","Data":"06c12150791b9a47df8a6cd2c2ffef0ed2da399b2bf41f2da5a4af2180a2c868"} Jul 02 23:07:54.915793 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:54.915766 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-tx9tm" event={"ID":"62b1e0d4-ca21-4def-85ea-e2f9b674834b","Type":"ContainerStarted","Data":"3bc3b69176ecb52f789848e6a7e49f04e8c4521dc630aac96f1b827d8e1d728d"} Jul 02 23:07:54.918819 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:54.918787 2583 generic.go:356] "Generic (PLEG): container finished" podID="511288f9-603d-46d3-aaf1-f1725e8f3226" containerID="8cddc97572a541a48433e9635ca6992dfaec18a05f00ced1235b8f4d9c98097c" exitCode=0 Jul 02 23:07:54.918950 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:54.918825 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xt994" event={"ID":"511288f9-603d-46d3-aaf1-f1725e8f3226","Type":"ContainerDied","Data":"8cddc97572a541a48433e9635ca6992dfaec18a05f00ced1235b8f4d9c98097c"} Jul 02 23:07:55.926103 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:55.926067 2583 generic.go:356] "Generic (PLEG): container finished" podID="511288f9-603d-46d3-aaf1-f1725e8f3226" containerID="4412ae6a46739fde9cb4eddf3ebf3dee70d2649c4dacc158de960fa069be2fb6" exitCode=0 Jul 02 23:07:55.926627 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:55.926119 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xt994" event={"ID":"511288f9-603d-46d3-aaf1-f1725e8f3226","Type":"ContainerDied","Data":"4412ae6a46739fde9cb4eddf3ebf3dee70d2649c4dacc158de960fa069be2fb6"} Jul 02 23:07:56.576185 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:56.576142 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:07:56.576185 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:56.576187 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-metrics-tls\") pod \"dns-default-2vvfs\" (UID: \"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79\") " pod="openshift-dns/dns-default-2vvfs" Jul 02 23:07:56.576404 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:56.576304 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:07:56.576404 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:56.576304 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:07:56.576404 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:56.576371 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-insights-runtime-extractor-tls podName:57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02 nodeName:}" failed. No retries permitted until 2026-07-02 23:08:00.576349847 +0000 UTC m=+44.599167938 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqx8m" (UID: "57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02") : secret "insights-runtime-extractor-tls" not found Jul 02 23:07:56.576404 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:07:56.576387 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-metrics-tls podName:fb8c01ab-cb1f-403b-aef4-c4a7143b7c79 nodeName:}" failed. No retries permitted until 2026-07-02 23:08:00.576380317 +0000 UTC m=+44.599198408 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-metrics-tls") pod "dns-default-2vvfs" (UID: "fb8c01ab-cb1f-403b-aef4-c4a7143b7c79") : secret "dns-default-metrics-tls" not found Jul 02 23:07:57.933408 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:57.933315 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vblxw" event={"ID":"952de78d-e0fb-4c00-88d6-ca117114fd67","Type":"ContainerStarted","Data":"381f4a57859c3409f845bc80ffe4e987e5fa633763f545e46d5df64da938a331"} Jul 02 23:07:57.934813 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:57.934786 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-tx9tm" event={"ID":"62b1e0d4-ca21-4def-85ea-e2f9b674834b","Type":"ContainerStarted","Data":"bb932c6fc67a07d16b008a56639886e3f56eafb30282def2f899987a5eef7184"} Jul 02 23:07:57.938152 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:57.938127 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xt994" event={"ID":"511288f9-603d-46d3-aaf1-f1725e8f3226","Type":"ContainerStarted","Data":"7dc1e4194263e6cdb9ec57c3dcc71bb6026d7a611635c836847c077692b3fc30"} Jul 02 23:07:57.953040 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:57.950684 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vblxw" podStartSLOduration=2.871791103 podStartE2EDuration="5.950665346s" totalImagesPullingTime="3.078874243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:07:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:07:57.949271256 +0000 UTC m=+41.972089371" watchObservedRunningTime="2026-07-02 23:07:57.950665346 +0000 UTC m=+41.973483459" Jul 02 23:07:57.966699 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:57.966634 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-tx9tm" podStartSLOduration=1.88682298 podStartE2EDuration="4.966617958s" totalImagesPullingTime="3.079794978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:07:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:07:57.966487661 +0000 UTC m=+41.989305770" watchObservedRunningTime="2026-07-02 23:07:57.966617958 +0000 UTC m=+41.989436070" Jul 02 23:07:57.988882 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:07:57.988826 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xt994" podStartSLOduration=7.5169122999999995 podStartE2EDuration="40.98880549s" totalImagesPullingTime="33.47189319s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:07:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:07:57.987203164 +0000 UTC m=+42.010021278" watchObservedRunningTime="2026-07-02 23:07:57.98880549 +0000 UTC m=+42.011623605" Jul 02 23:08:00.609884 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:00.609842 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:08:00.609884 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:00.609884 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-metrics-tls\") pod \"dns-default-2vvfs\" (UID: \"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79\") " pod="openshift-dns/dns-default-2vvfs" Jul 02 23:08:00.610311 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:08:00.609999 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:08:00.610311 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:08:00.610004 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:08:00.610311 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:08:00.610052 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-metrics-tls podName:fb8c01ab-cb1f-403b-aef4-c4a7143b7c79 nodeName:}" failed. No retries permitted until 2026-07-02 23:08:08.610038887 +0000 UTC m=+52.632856978 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-metrics-tls") pod "dns-default-2vvfs" (UID: "fb8c01ab-cb1f-403b-aef4-c4a7143b7c79") : secret "dns-default-metrics-tls" not found Jul 02 23:08:00.610311 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:08:00.610067 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-insights-runtime-extractor-tls podName:57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02 nodeName:}" failed. No retries permitted until 2026-07-02 23:08:08.61006056 +0000 UTC m=+52.632878651 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqx8m" (UID: "57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02") : secret "insights-runtime-extractor-tls" not found Jul 02 23:08:08.670574 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:08.670514 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:08:08.670574 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:08.670571 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-metrics-tls\") pod \"dns-default-2vvfs\" (UID: \"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79\") " pod="openshift-dns/dns-default-2vvfs" Jul 02 23:08:08.674099 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:08.674071 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fb8c01ab-cb1f-403b-aef4-c4a7143b7c79-metrics-tls\") pod \"dns-default-2vvfs\" (UID: \"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79\") " pod="openshift-dns/dns-default-2vvfs" Jul 02 23:08:08.674215 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:08.674077 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqx8m\" (UID: \"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02\") " pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:08:08.692996 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:08.692964 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bqx8m" Jul 02 23:08:08.703116 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:08.703083 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2vvfs" Jul 02 23:08:08.848591 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:08.848563 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bqx8m"] Jul 02 23:08:08.870670 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:08.870644 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2vvfs"] Jul 02 23:08:08.873650 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:08:08.873627 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb8c01ab_cb1f_403b_aef4_c4a7143b7c79.slice/crio-6f9342bf0759e6e7cf32709b5b7177b42f67cbe792e30ee47e4e5ffe10578308 WatchSource:0}: Error finding container 6f9342bf0759e6e7cf32709b5b7177b42f67cbe792e30ee47e4e5ffe10578308: Status 404 returned error can't find the container with id 6f9342bf0759e6e7cf32709b5b7177b42f67cbe792e30ee47e4e5ffe10578308 Jul 02 23:08:08.967073 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:08.966998 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqx8m" event={"ID":"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02","Type":"ContainerStarted","Data":"aa7206ccf81e287856a050db32cfdd8eb86e6b67bd54a346642f2b131eaa557d"} Jul 02 23:08:08.968002 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:08.967977 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2vvfs" event={"ID":"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79","Type":"ContainerStarted","Data":"6f9342bf0759e6e7cf32709b5b7177b42f67cbe792e30ee47e4e5ffe10578308"} Jul 02 23:08:13.986874 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:13.986781 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2vvfs" event={"ID":"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79","Type":"ContainerStarted","Data":"7f7661418768d73a2721b951be4f4e033b9426ef8fd2b6090b70d866b4712f5f"} Jul 02 23:08:13.986874 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:13.986868 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2vvfs" event={"ID":"fb8c01ab-cb1f-403b-aef4-c4a7143b7c79","Type":"ContainerStarted","Data":"191b2c08891ab9516065fe3203ab5f66886a21a8140a03c9867192647178dcff"} Jul 02 23:08:13.987354 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:13.986914 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2vvfs" Jul 02 23:08:13.988580 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:13.988553 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqx8m" event={"ID":"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02","Type":"ContainerStarted","Data":"b789282e9ff6bde58d76bb7d82c1932104ae464e8742fd1a1fe0b7fa9de9bbd4"} Jul 02 23:08:13.988678 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:13.988608 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqx8m" event={"ID":"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02","Type":"ContainerStarted","Data":"dbc98c3e007b19b79a02201d6421e98392948503eb2a0b0028828cfeb85d8c08"} Jul 02 23:08:15.512635 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.512574 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2vvfs" podStartSLOduration=19.407163447 podStartE2EDuration="23.512555419s" totalImagesPullingTime="4.105391972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:07:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:08:14.002166793 +0000 UTC m=+58.024984907" watchObservedRunningTime="2026-07-02 23:08:15.512555419 +0000 UTC m=+59.535373544" Jul 02 23:08:15.513068 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.512708 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b59d8db98-zkxrg"] Jul 02 23:08:15.519208 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.519171 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448"] Jul 02 23:08:15.519369 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.519310 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b59d8db98-zkxrg" Jul 02 23:08:15.521537 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.521503 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 02 23:08:15.522133 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.521831 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 02 23:08:15.522133 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.521936 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 02 23:08:15.522297 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.522247 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:15.522388 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.522373 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 02 23:08:15.522993 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.522971 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-24n2p\"" Jul 02 23:08:15.524264 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.524243 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 02 23:08:15.524419 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.524395 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 02 23:08:15.524864 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.524621 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 02 23:08:15.524864 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.524720 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 02 23:08:15.540847 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.540816 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b59d8db98-zkxrg"] Jul 02 23:08:15.541989 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.541953 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448"] Jul 02 23:08:15.615494 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.615458 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9e11e559-b7b7-4d5e-87ac-ba664f811389-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b59d8db98-zkxrg\" (UID: \"9e11e559-b7b7-4d5e-87ac-ba664f811389\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b59d8db98-zkxrg" Jul 02 23:08:15.615494 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.615499 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/730dfd32-b3ea-4b30-87e0-4ad78eaac6a6-hub\") pod \"cluster-proxy-proxy-agent-dbf8646f5-4k448\" (UID: \"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:15.615730 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.615525 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/730dfd32-b3ea-4b30-87e0-4ad78eaac6a6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-dbf8646f5-4k448\" (UID: \"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:15.615730 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.615561 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkfmt\" (UniqueName: \"kubernetes.io/projected/730dfd32-b3ea-4b30-87e0-4ad78eaac6a6-kube-api-access-zkfmt\") pod \"cluster-proxy-proxy-agent-dbf8646f5-4k448\" (UID: \"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:15.615730 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.615591 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/730dfd32-b3ea-4b30-87e0-4ad78eaac6a6-ca\") pod \"cluster-proxy-proxy-agent-dbf8646f5-4k448\" (UID: \"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:15.615730 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.615637 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh275\" (UniqueName: \"kubernetes.io/projected/9e11e559-b7b7-4d5e-87ac-ba664f811389-kube-api-access-kh275\") pod \"managed-serviceaccount-addon-agent-6b59d8db98-zkxrg\" (UID: \"9e11e559-b7b7-4d5e-87ac-ba664f811389\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b59d8db98-zkxrg" Jul 02 23:08:15.615730 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.615696 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/730dfd32-b3ea-4b30-87e0-4ad78eaac6a6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-dbf8646f5-4k448\" (UID: \"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:15.615949 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.615750 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/730dfd32-b3ea-4b30-87e0-4ad78eaac6a6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-dbf8646f5-4k448\" (UID: \"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:15.716159 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.716125 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9e11e559-b7b7-4d5e-87ac-ba664f811389-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b59d8db98-zkxrg\" (UID: \"9e11e559-b7b7-4d5e-87ac-ba664f811389\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b59d8db98-zkxrg" Jul 02 23:08:15.716159 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.716163 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/730dfd32-b3ea-4b30-87e0-4ad78eaac6a6-hub\") pod \"cluster-proxy-proxy-agent-dbf8646f5-4k448\" (UID: \"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:15.716410 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.716193 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/730dfd32-b3ea-4b30-87e0-4ad78eaac6a6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-dbf8646f5-4k448\" (UID: \"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:15.716410 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.716366 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkfmt\" (UniqueName: \"kubernetes.io/projected/730dfd32-b3ea-4b30-87e0-4ad78eaac6a6-kube-api-access-zkfmt\") pod \"cluster-proxy-proxy-agent-dbf8646f5-4k448\" (UID: \"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:15.716496 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.716413 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/730dfd32-b3ea-4b30-87e0-4ad78eaac6a6-ca\") pod \"cluster-proxy-proxy-agent-dbf8646f5-4k448\" (UID: \"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:15.716496 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.716472 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh275\" (UniqueName: \"kubernetes.io/projected/9e11e559-b7b7-4d5e-87ac-ba664f811389-kube-api-access-kh275\") pod \"managed-serviceaccount-addon-agent-6b59d8db98-zkxrg\" (UID: \"9e11e559-b7b7-4d5e-87ac-ba664f811389\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b59d8db98-zkxrg" Jul 02 23:08:15.716606 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.716513 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/730dfd32-b3ea-4b30-87e0-4ad78eaac6a6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-dbf8646f5-4k448\" (UID: \"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:15.716606 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.716551 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/730dfd32-b3ea-4b30-87e0-4ad78eaac6a6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-dbf8646f5-4k448\" (UID: \"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:15.717693 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.717634 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/730dfd32-b3ea-4b30-87e0-4ad78eaac6a6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-dbf8646f5-4k448\" (UID: \"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:15.719368 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.719333 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/730dfd32-b3ea-4b30-87e0-4ad78eaac6a6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-dbf8646f5-4k448\" (UID: \"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:15.719479 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.719444 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/730dfd32-b3ea-4b30-87e0-4ad78eaac6a6-hub\") pod \"cluster-proxy-proxy-agent-dbf8646f5-4k448\" (UID: \"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:15.719540 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.719484 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9e11e559-b7b7-4d5e-87ac-ba664f811389-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b59d8db98-zkxrg\" (UID: \"9e11e559-b7b7-4d5e-87ac-ba664f811389\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b59d8db98-zkxrg" Jul 02 23:08:15.719769 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.719749 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/730dfd32-b3ea-4b30-87e0-4ad78eaac6a6-ca\") pod \"cluster-proxy-proxy-agent-dbf8646f5-4k448\" (UID: \"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:15.720571 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.720546 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/730dfd32-b3ea-4b30-87e0-4ad78eaac6a6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-dbf8646f5-4k448\" (UID: \"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:15.730591 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.728507 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh275\" (UniqueName: \"kubernetes.io/projected/9e11e559-b7b7-4d5e-87ac-ba664f811389-kube-api-access-kh275\") pod \"managed-serviceaccount-addon-agent-6b59d8db98-zkxrg\" (UID: \"9e11e559-b7b7-4d5e-87ac-ba664f811389\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b59d8db98-zkxrg" Jul 02 23:08:15.730591 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.728868 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkfmt\" (UniqueName: \"kubernetes.io/projected/730dfd32-b3ea-4b30-87e0-4ad78eaac6a6-kube-api-access-zkfmt\") pod \"cluster-proxy-proxy-agent-dbf8646f5-4k448\" (UID: \"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:15.841457 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.841423 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b59d8db98-zkxrg" Jul 02 23:08:15.848726 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:15.848700 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" Jul 02 23:08:16.219752 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:16.219717 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448"] Jul 02 23:08:16.223039 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:08:16.223013 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod730dfd32_b3ea_4b30_87e0_4ad78eaac6a6.slice/crio-0aa2748dfeeba989a4f4ec81f324def8946a9f2713e1673df5cac91f6c62f769 WatchSource:0}: Error finding container 0aa2748dfeeba989a4f4ec81f324def8946a9f2713e1673df5cac91f6c62f769: Status 404 returned error can't find the container with id 0aa2748dfeeba989a4f4ec81f324def8946a9f2713e1673df5cac91f6c62f769 Jul 02 23:08:16.235439 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:16.235411 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b59d8db98-zkxrg"] Jul 02 23:08:16.238716 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:08:16.238691 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e11e559_b7b7_4d5e_87ac_ba664f811389.slice/crio-6860efa144874b8395fe7ee366a9617593be7cd75721869014d82dccbe58ace2 WatchSource:0}: Error finding container 6860efa144874b8395fe7ee366a9617593be7cd75721869014d82dccbe58ace2: Status 404 returned error can't find the container with id 6860efa144874b8395fe7ee366a9617593be7cd75721869014d82dccbe58ace2 Jul 02 23:08:17.003262 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:17.003213 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqx8m" event={"ID":"57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02","Type":"ContainerStarted","Data":"72008e677c6535a5e8f60ad9f6d93c6a9e243f8fbc59d610251ef40aadab3736"} Jul 02 23:08:17.005518 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:17.005482 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" event={"ID":"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6","Type":"ContainerStarted","Data":"0aa2748dfeeba989a4f4ec81f324def8946a9f2713e1673df5cac91f6c62f769"} Jul 02 23:08:17.007417 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:17.007387 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b59d8db98-zkxrg" event={"ID":"9e11e559-b7b7-4d5e-87ac-ba664f811389","Type":"ContainerStarted","Data":"6860efa144874b8395fe7ee366a9617593be7cd75721869014d82dccbe58ace2"} Jul 02 23:08:17.021505 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:17.021444 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bqx8m" podStartSLOduration=17.876593882999998 podStartE2EDuration="25.021424997s" totalImagesPullingTime="7.144831114s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:07:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:08:17.020693904 +0000 UTC m=+61.043512018" watchObservedRunningTime="2026-07-02 23:08:17.021424997 +0000 UTC m=+61.044243111" Jul 02 23:08:18.908695 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:18.908666 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fcv6k" Jul 02 23:08:20.023194 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:20.023145 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b59d8db98-zkxrg" event={"ID":"9e11e559-b7b7-4d5e-87ac-ba664f811389","Type":"ContainerStarted","Data":"30f224563843c219e84c7d7684cab76e87b867370f5fb789dd6ee57b9704c79d"} Jul 02 23:08:20.040606 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:20.040546 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b59d8db98-zkxrg" podStartSLOduration=2.23004301 podStartE2EDuration="5.040526686s" totalImagesPullingTime="2.810483676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:08:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:08:20.040107479 +0000 UTC m=+64.062925604" watchObservedRunningTime="2026-07-02 23:08:20.040526686 +0000 UTC m=+64.063344801" Jul 02 23:08:21.028248 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.028209 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" event={"ID":"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6","Type":"ContainerStarted","Data":"f6cb8062d7e513c611478b80697d4f2ffc221153694a91f3c32d0e8974543b64"} Jul 02 23:08:21.028248 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.028255 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" event={"ID":"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6","Type":"ContainerStarted","Data":"feab0ecd1ddb1bd5dc2094f2ad4a1c57bea130a4eb0a1b43161dfc6c99a98ca7"} Jul 02 23:08:21.028753 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.028269 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" event={"ID":"730dfd32-b3ea-4b30-87e0-4ad78eaac6a6","Type":"ContainerStarted","Data":"b812787d5507c6984447907d01015cf006443efd1634b8b1bd5937f0c759b677"} Jul 02 23:08:21.047201 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.047149 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dbf8646f5-4k448" podStartSLOduration=1.750971243 podStartE2EDuration="6.047133983s" totalImagesPullingTime="4.29616274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:08:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:08:21.045802823 +0000 UTC m=+65.068620949" watchObservedRunningTime="2026-07-02 23:08:21.047133983 +0000 UTC m=+65.069952122" Jul 02 23:08:21.263941 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.263859 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs\") pod \"network-metrics-daemon-xqwp5\" (UID: \"e3f048d9-2670-488f-9a21-73f967c069b7\") " pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:08:21.264145 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.263974 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret\") pod \"global-pull-secret-syncer-rs2g8\" (UID: \"dcb0135c-18a2-4e25-9193-16f4e2a3b833\") " pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:08:21.266517 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.266495 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 23:08:21.266583 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.266519 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 23:08:21.276734 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.276701 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb0135c-18a2-4e25-9193-16f4e2a3b833-original-pull-secret\") pod \"global-pull-secret-syncer-rs2g8\" (UID: \"dcb0135c-18a2-4e25-9193-16f4e2a3b833\") " pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:08:21.276849 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.276751 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3f048d9-2670-488f-9a21-73f967c069b7-metrics-certs\") pod \"network-metrics-daemon-xqwp5\" (UID: \"e3f048d9-2670-488f-9a21-73f967c069b7\") " pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:08:21.364470 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.364432 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzfdj\" (UniqueName: \"kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj\") pod \"network-check-target-drz87\" (UID: \"4431a02d-4597-4cb5-ad47-faaa53a10233\") " pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:08:21.367061 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.367039 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 23:08:21.376840 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.376819 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 23:08:21.388158 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.388134 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzfdj\" (UniqueName: \"kubernetes.io/projected/4431a02d-4597-4cb5-ad47-faaa53a10233-kube-api-access-kzfdj\") pod \"network-check-target-drz87\" (UID: \"4431a02d-4597-4cb5-ad47-faaa53a10233\") " pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:08:21.440786 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.440755 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dptv4\"" Jul 02 23:08:21.448380 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.448356 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jkmkq\"" Jul 02 23:08:21.449386 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.449371 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:08:21.452466 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.452454 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rs2g8" Jul 02 23:08:21.456784 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.456763 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqwp5" Jul 02 23:08:21.594415 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.594338 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-drz87"] Jul 02 23:08:21.598965 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:08:21.598923 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4431a02d_4597_4cb5_ad47_faaa53a10233.slice/crio-d0882762c14277f083c2bea8645cf50a7fd352009521812aaf248eb1373cd2c7 WatchSource:0}: Error finding container d0882762c14277f083c2bea8645cf50a7fd352009521812aaf248eb1373cd2c7: Status 404 returned error can't find the container with id d0882762c14277f083c2bea8645cf50a7fd352009521812aaf248eb1373cd2c7 Jul 02 23:08:21.821288 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.821237 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rs2g8"] Jul 02 23:08:21.822109 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:21.822083 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xqwp5"] Jul 02 23:08:21.824550 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:08:21.824522 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcb0135c_18a2_4e25_9193_16f4e2a3b833.slice/crio-6367eb3dd5f5f123312152ce987d6f0a7c12f3b402e8ba036c4a0603e40ba336 WatchSource:0}: Error finding container 6367eb3dd5f5f123312152ce987d6f0a7c12f3b402e8ba036c4a0603e40ba336: Status 404 returned error can't find the container with id 6367eb3dd5f5f123312152ce987d6f0a7c12f3b402e8ba036c4a0603e40ba336 Jul 02 23:08:21.825341 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:08:21.825315 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3f048d9_2670_488f_9a21_73f967c069b7.slice/crio-ef558871d959463a0c02d70c6445e64ddb0cf3ddc17e05779bee8f7785d3eab6 WatchSource:0}: Error finding container ef558871d959463a0c02d70c6445e64ddb0cf3ddc17e05779bee8f7785d3eab6: Status 404 returned error can't find the container with id ef558871d959463a0c02d70c6445e64ddb0cf3ddc17e05779bee8f7785d3eab6 Jul 02 23:08:22.033433 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:22.033396 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-drz87" event={"ID":"4431a02d-4597-4cb5-ad47-faaa53a10233","Type":"ContainerStarted","Data":"d0882762c14277f083c2bea8645cf50a7fd352009521812aaf248eb1373cd2c7"} Jul 02 23:08:22.034808 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:22.034776 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xqwp5" event={"ID":"e3f048d9-2670-488f-9a21-73f967c069b7","Type":"ContainerStarted","Data":"ef558871d959463a0c02d70c6445e64ddb0cf3ddc17e05779bee8f7785d3eab6"} Jul 02 23:08:22.036161 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:22.036134 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rs2g8" event={"ID":"dcb0135c-18a2-4e25-9193-16f4e2a3b833","Type":"ContainerStarted","Data":"6367eb3dd5f5f123312152ce987d6f0a7c12f3b402e8ba036c4a0603e40ba336"} Jul 02 23:08:23.042423 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:23.042389 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xqwp5" event={"ID":"e3f048d9-2670-488f-9a21-73f967c069b7","Type":"ContainerStarted","Data":"a196cce9805697055c5e756638d29cfad8ba59e9275e4924ebfc796fc00257c7"} Jul 02 23:08:23.994814 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:23.994783 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2vvfs" Jul 02 23:08:24.049176 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:24.049140 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xqwp5" event={"ID":"e3f048d9-2670-488f-9a21-73f967c069b7","Type":"ContainerStarted","Data":"d1ad4bcabb5e130b4f5992c9854fb67ea8457e61ae05bebb0554a2ff4e1f191d"} Jul 02 23:08:24.064251 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:24.064175 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xqwp5" podStartSLOduration=65.982959946 podStartE2EDuration="1m7.064153774s" totalImagesPullingTime="1.081193828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:08:24.063929405 +0000 UTC m=+68.086747520" watchObservedRunningTime="2026-07-02 23:08:24.064153774 +0000 UTC m=+68.086971890" Jul 02 23:08:27.060936 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:27.060898 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rs2g8" event={"ID":"dcb0135c-18a2-4e25-9193-16f4e2a3b833","Type":"ContainerStarted","Data":"745a9e1c63fe566c522a1f8be1c4332fc59dc550c47c2880629478db7d4f8041"} Jul 02 23:08:27.065623 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:27.062465 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-drz87" event={"ID":"4431a02d-4597-4cb5-ad47-faaa53a10233","Type":"ContainerStarted","Data":"afbd0739431a5a18c0ac9f734589a50f06b6ab543f47cb4ec07637420c75e259"} Jul 02 23:08:27.065623 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:27.063007 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:08:27.077781 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:27.077739 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rs2g8" podStartSLOduration=65.279103333 podStartE2EDuration="1m10.077725089s" totalImagesPullingTime="4.798621756s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:08:27.076456231 +0000 UTC m=+71.099274344" watchObservedRunningTime="2026-07-02 23:08:27.077725089 +0000 UTC m=+71.100543200" Jul 02 23:08:27.090047 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:27.090003 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-drz87" podStartSLOduration=65.070201927 podStartE2EDuration="1m10.089986122s" totalImagesPullingTime="5.019784195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:08:27.089158154 +0000 UTC m=+71.111976260" watchObservedRunningTime="2026-07-02 23:08:27.089986122 +0000 UTC m=+71.112804234" Jul 02 23:08:40.508980 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.508829 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj"] Jul 02 23:08:40.512057 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.512034 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" Jul 02 23:08:40.514596 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.514570 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 02 23:08:40.514716 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.514616 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 23:08:40.514811 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.514588 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 23:08:40.514935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.514839 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-jqqqm\"" Jul 02 23:08:40.515460 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.515444 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 02 23:08:40.515550 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.515485 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 23:08:40.526262 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.526228 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj"] Jul 02 23:08:40.543849 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.543811 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-f5pkx"] Jul 02 23:08:40.546909 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.546868 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.549917 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.549877 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 23:08:40.550365 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.550335 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 23:08:40.550735 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.550709 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 23:08:40.550930 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.550525 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tk5z5\"" Jul 02 23:08:40.605111 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.605073 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f2910e39-d10c-46c7-ac46-01534729dff7-node-exporter-accelerators-collector-config\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.605111 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.605116 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2910e39-d10c-46c7-ac46-01534729dff7-metrics-client-ca\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.605366 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.605142 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2910e39-d10c-46c7-ac46-01534729dff7-node-exporter-tls\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.605366 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.605197 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2910e39-d10c-46c7-ac46-01534729dff7-sys\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.605366 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.605243 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2910e39-d10c-46c7-ac46-01534729dff7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.605366 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.605260 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f2910e39-d10c-46c7-ac46-01534729dff7-node-exporter-textfile\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.605366 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.605302 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd4a33d2-1122-4c89-8d4b-2c4b901e03b9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-brftj\" (UID: \"bd4a33d2-1122-4c89-8d4b-2c4b901e03b9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" Jul 02 23:08:40.605366 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.605327 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f2910e39-d10c-46c7-ac46-01534729dff7-root\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.605708 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.605407 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd4a33d2-1122-4c89-8d4b-2c4b901e03b9-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-brftj\" (UID: \"bd4a33d2-1122-4c89-8d4b-2c4b901e03b9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" Jul 02 23:08:40.605708 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.605452 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f2910e39-d10c-46c7-ac46-01534729dff7-node-exporter-wtmp\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.605708 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.605482 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnb7k\" (UniqueName: \"kubernetes.io/projected/f2910e39-d10c-46c7-ac46-01534729dff7-kube-api-access-mnb7k\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.605708 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.605510 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd4a33d2-1122-4c89-8d4b-2c4b901e03b9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-brftj\" (UID: \"bd4a33d2-1122-4c89-8d4b-2c4b901e03b9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" Jul 02 23:08:40.605708 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.605538 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrlq7\" (UniqueName: \"kubernetes.io/projected/bd4a33d2-1122-4c89-8d4b-2c4b901e03b9-kube-api-access-rrlq7\") pod \"openshift-state-metrics-54cf96dfcf-brftj\" (UID: \"bd4a33d2-1122-4c89-8d4b-2c4b901e03b9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" Jul 02 23:08:40.706051 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.706008 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f2910e39-d10c-46c7-ac46-01534729dff7-node-exporter-wtmp\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.706051 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.706050 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnb7k\" (UniqueName: \"kubernetes.io/projected/f2910e39-d10c-46c7-ac46-01534729dff7-kube-api-access-mnb7k\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.706317 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.706069 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd4a33d2-1122-4c89-8d4b-2c4b901e03b9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-brftj\" (UID: \"bd4a33d2-1122-4c89-8d4b-2c4b901e03b9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" Jul 02 23:08:40.706317 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.706196 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrlq7\" (UniqueName: \"kubernetes.io/projected/bd4a33d2-1122-4c89-8d4b-2c4b901e03b9-kube-api-access-rrlq7\") pod \"openshift-state-metrics-54cf96dfcf-brftj\" (UID: \"bd4a33d2-1122-4c89-8d4b-2c4b901e03b9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" Jul 02 23:08:40.706317 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.706228 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f2910e39-d10c-46c7-ac46-01534729dff7-node-exporter-wtmp\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.706317 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.706250 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f2910e39-d10c-46c7-ac46-01534729dff7-node-exporter-accelerators-collector-config\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.706317 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.706292 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2910e39-d10c-46c7-ac46-01534729dff7-metrics-client-ca\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.706581 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.706319 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2910e39-d10c-46c7-ac46-01534729dff7-node-exporter-tls\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.706581 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.706348 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2910e39-d10c-46c7-ac46-01534729dff7-sys\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.706581 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.706374 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2910e39-d10c-46c7-ac46-01534729dff7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.706581 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.706398 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f2910e39-d10c-46c7-ac46-01534729dff7-node-exporter-textfile\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.706581 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:08:40.706460 2583 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 02 23:08:40.706581 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:08:40.706533 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2910e39-d10c-46c7-ac46-01534729dff7-node-exporter-tls podName:f2910e39-d10c-46c7-ac46-01534729dff7 nodeName:}" failed. No retries permitted until 2026-07-02 23:08:41.206503291 +0000 UTC m=+85.229321403 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f2910e39-d10c-46c7-ac46-01534729dff7-node-exporter-tls") pod "node-exporter-f5pkx" (UID: "f2910e39-d10c-46c7-ac46-01534729dff7") : secret "node-exporter-tls" not found Jul 02 23:08:40.706883 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.706460 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd4a33d2-1122-4c89-8d4b-2c4b901e03b9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-brftj\" (UID: \"bd4a33d2-1122-4c89-8d4b-2c4b901e03b9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" Jul 02 23:08:40.706883 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.706753 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f2910e39-d10c-46c7-ac46-01534729dff7-root\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.706883 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.706797 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd4a33d2-1122-4c89-8d4b-2c4b901e03b9-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-brftj\" (UID: \"bd4a33d2-1122-4c89-8d4b-2c4b901e03b9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" Jul 02 23:08:40.706883 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.706867 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f2910e39-d10c-46c7-ac46-01534729dff7-node-exporter-accelerators-collector-config\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.707118 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.707004 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2910e39-d10c-46c7-ac46-01534729dff7-metrics-client-ca\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.707118 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.707084 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2910e39-d10c-46c7-ac46-01534729dff7-sys\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.707215 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.707127 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f2910e39-d10c-46c7-ac46-01534729dff7-root\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.707215 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:08:40.707151 2583 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 02 23:08:40.707308 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.707209 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f2910e39-d10c-46c7-ac46-01534729dff7-node-exporter-textfile\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.707308 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:08:40.707227 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd4a33d2-1122-4c89-8d4b-2c4b901e03b9-openshift-state-metrics-tls podName:bd4a33d2-1122-4c89-8d4b-2c4b901e03b9 nodeName:}" failed. No retries permitted until 2026-07-02 23:08:41.207210317 +0000 UTC m=+85.230028409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/bd4a33d2-1122-4c89-8d4b-2c4b901e03b9-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-brftj" (UID: "bd4a33d2-1122-4c89-8d4b-2c4b901e03b9") : secret "openshift-state-metrics-tls" not found Jul 02 23:08:40.707535 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.707515 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd4a33d2-1122-4c89-8d4b-2c4b901e03b9-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-brftj\" (UID: \"bd4a33d2-1122-4c89-8d4b-2c4b901e03b9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" Jul 02 23:08:40.709510 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.709491 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd4a33d2-1122-4c89-8d4b-2c4b901e03b9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-brftj\" (UID: \"bd4a33d2-1122-4c89-8d4b-2c4b901e03b9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" Jul 02 23:08:40.710169 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.710143 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2910e39-d10c-46c7-ac46-01534729dff7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.714748 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.714726 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnb7k\" (UniqueName: \"kubernetes.io/projected/f2910e39-d10c-46c7-ac46-01534729dff7-kube-api-access-mnb7k\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:40.715255 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:40.715235 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrlq7\" (UniqueName: \"kubernetes.io/projected/bd4a33d2-1122-4c89-8d4b-2c4b901e03b9-kube-api-access-rrlq7\") pod \"openshift-state-metrics-54cf96dfcf-brftj\" (UID: \"bd4a33d2-1122-4c89-8d4b-2c4b901e03b9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" Jul 02 23:08:41.211332 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:41.211295 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd4a33d2-1122-4c89-8d4b-2c4b901e03b9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-brftj\" (UID: \"bd4a33d2-1122-4c89-8d4b-2c4b901e03b9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" Jul 02 23:08:41.211514 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:41.211342 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2910e39-d10c-46c7-ac46-01534729dff7-node-exporter-tls\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:41.213956 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:41.213930 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2910e39-d10c-46c7-ac46-01534729dff7-node-exporter-tls\") pod \"node-exporter-f5pkx\" (UID: \"f2910e39-d10c-46c7-ac46-01534729dff7\") " pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:41.214062 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:41.213971 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd4a33d2-1122-4c89-8d4b-2c4b901e03b9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-brftj\" (UID: \"bd4a33d2-1122-4c89-8d4b-2c4b901e03b9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" Jul 02 23:08:41.423433 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:41.423374 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" Jul 02 23:08:41.461966 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:41.461803 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f5pkx" Jul 02 23:08:41.475572 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:08:41.475542 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2910e39_d10c_46c7_ac46_01534729dff7.slice/crio-5b872dbe17276147f6834e0bc09b4d1c921f9d15fbbb451f0ec5cd9dbdafea59 WatchSource:0}: Error finding container 5b872dbe17276147f6834e0bc09b4d1c921f9d15fbbb451f0ec5cd9dbdafea59: Status 404 returned error can't find the container with id 5b872dbe17276147f6834e0bc09b4d1c921f9d15fbbb451f0ec5cd9dbdafea59 Jul 02 23:08:41.586313 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:41.586285 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj"] Jul 02 23:08:41.589656 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:08:41.589629 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd4a33d2_1122_4c89_8d4b_2c4b901e03b9.slice/crio-9ef6ec32354b1d0a6ba7f43b27e649340301da9ccabb473ac582c9840ac55abc WatchSource:0}: Error finding container 9ef6ec32354b1d0a6ba7f43b27e649340301da9ccabb473ac582c9840ac55abc: Status 404 returned error can't find the container with id 9ef6ec32354b1d0a6ba7f43b27e649340301da9ccabb473ac582c9840ac55abc Jul 02 23:08:42.110755 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:42.110719 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f5pkx" event={"ID":"f2910e39-d10c-46c7-ac46-01534729dff7","Type":"ContainerStarted","Data":"5b872dbe17276147f6834e0bc09b4d1c921f9d15fbbb451f0ec5cd9dbdafea59"} Jul 02 23:08:42.113398 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:42.113360 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" event={"ID":"bd4a33d2-1122-4c89-8d4b-2c4b901e03b9","Type":"ContainerStarted","Data":"7821a8217a08207cbc15879c3ec5d3861a212c8b00b46f384e659a6641e6065f"} Jul 02 23:08:42.113552 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:42.113434 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" event={"ID":"bd4a33d2-1122-4c89-8d4b-2c4b901e03b9","Type":"ContainerStarted","Data":"b354002bc0a63d94a1371fcb3896b373d628fb83e367443d3c5a63b58c42f092"} Jul 02 23:08:42.113552 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:42.113464 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" event={"ID":"bd4a33d2-1122-4c89-8d4b-2c4b901e03b9","Type":"ContainerStarted","Data":"9ef6ec32354b1d0a6ba7f43b27e649340301da9ccabb473ac582c9840ac55abc"} Jul 02 23:08:43.118615 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:43.118578 2583 generic.go:356] "Generic (PLEG): container finished" podID="f2910e39-d10c-46c7-ac46-01534729dff7" containerID="d7e359f58bc493eb6b5835c47b1c8712da7cb8523ad1c510e854171d818d2142" exitCode=0 Jul 02 23:08:43.119071 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:43.118669 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f5pkx" event={"ID":"f2910e39-d10c-46c7-ac46-01534729dff7","Type":"ContainerDied","Data":"d7e359f58bc493eb6b5835c47b1c8712da7cb8523ad1c510e854171d818d2142"} Jul 02 23:08:43.120990 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:43.120965 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" event={"ID":"bd4a33d2-1122-4c89-8d4b-2c4b901e03b9","Type":"ContainerStarted","Data":"538f4e23f4a31b3193c6ed3f0312a721411b34b426a0e10f7f9b6e8005d9bae2"} Jul 02 23:08:43.155915 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:43.155845 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-brftj" podStartSLOduration=2.100607809 podStartE2EDuration="3.155829536s" totalImagesPullingTime="1.055221727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:08:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:08:43.154868598 +0000 UTC m=+87.177686711" watchObservedRunningTime="2026-07-02 23:08:43.155829536 +0000 UTC m=+87.178647667" Jul 02 23:08:44.130909 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:44.130853 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f5pkx" event={"ID":"f2910e39-d10c-46c7-ac46-01534729dff7","Type":"ContainerStarted","Data":"67dbcf7b38efb308786c048302458e25322ae4a486ac50d1f478a141f5aae4ec"} Jul 02 23:08:44.131393 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:44.130966 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f5pkx" event={"ID":"f2910e39-d10c-46c7-ac46-01534729dff7","Type":"ContainerStarted","Data":"71c3e8c17416214bfe81b6ddefb2b1270ed43b45d3fbd85fc8750727f068a111"} Jul 02 23:08:44.150252 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:44.150200 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-f5pkx" podStartSLOduration=3.046981086 podStartE2EDuration="4.150182981s" totalImagesPullingTime="1.027842369s" totalInitContainerRuntime="75.359526ms" isStatefulPod=true podCreationTimestamp="2026-07-02 23:08:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:08:44.148858874 +0000 UTC m=+88.171676989" watchObservedRunningTime="2026-07-02 23:08:44.150182981 +0000 UTC m=+88.173001094" Jul 02 23:08:47.112879 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.112835 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5bf4774669-vqxdg"] Jul 02 23:08:47.128583 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.128545 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bf4774669-vqxdg"] Jul 02 23:08:47.128753 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.128699 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.131078 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.131053 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vmc2w\"" Jul 02 23:08:47.131222 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.131078 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 02 23:08:47.131222 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.131082 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 02 23:08:47.131222 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.131085 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 02 23:08:47.131856 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.131752 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 02 23:08:47.131856 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.131796 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 02 23:08:47.131856 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.131833 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 02 23:08:47.131856 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.131833 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 02 23:08:47.136741 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.136723 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 02 23:08:47.260035 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.260003 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-service-ca\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.260035 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.260040 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d64e645-f005-4b2d-8949-50043bf093f3-console-oauth-config\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.260237 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.260060 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-console-config\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.260237 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.260166 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d64e645-f005-4b2d-8949-50043bf093f3-console-serving-cert\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.260237 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.260197 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-oauth-serving-cert\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.260237 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.260231 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk644\" (UniqueName: \"kubernetes.io/projected/7d64e645-f005-4b2d-8949-50043bf093f3-kube-api-access-fk644\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.260360 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.260271 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-trusted-ca-bundle\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.361181 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.361145 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-trusted-ca-bundle\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.361400 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.361194 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-service-ca\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.361400 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.361226 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d64e645-f005-4b2d-8949-50043bf093f3-console-oauth-config\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.361400 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.361257 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-console-config\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.361400 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.361295 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d64e645-f005-4b2d-8949-50043bf093f3-console-serving-cert\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.361600 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.361412 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-oauth-serving-cert\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.361600 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.361532 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk644\" (UniqueName: \"kubernetes.io/projected/7d64e645-f005-4b2d-8949-50043bf093f3-kube-api-access-fk644\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.362182 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.362155 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-service-ca\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.362293 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.362217 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-trusted-ca-bundle\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.362371 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.362352 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-oauth-serving-cert\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.362439 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.362374 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-console-config\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.364047 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.363998 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d64e645-f005-4b2d-8949-50043bf093f3-console-oauth-config\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.364254 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.364237 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d64e645-f005-4b2d-8949-50043bf093f3-console-serving-cert\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.369921 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.369869 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk644\" (UniqueName: \"kubernetes.io/projected/7d64e645-f005-4b2d-8949-50043bf093f3-kube-api-access-fk644\") pod \"console-5bf4774669-vqxdg\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.439444 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.439404 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:47.572714 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:47.572676 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bf4774669-vqxdg"] Jul 02 23:08:47.575823 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:08:47.575794 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d64e645_f005_4b2d_8949_50043bf093f3.slice/crio-e007e581f93ae98f657a6fd6452c2428d13501c66c940bb9a6a04c37ea8bec6b WatchSource:0}: Error finding container e007e581f93ae98f657a6fd6452c2428d13501c66c940bb9a6a04c37ea8bec6b: Status 404 returned error can't find the container with id e007e581f93ae98f657a6fd6452c2428d13501c66c940bb9a6a04c37ea8bec6b Jul 02 23:08:48.146539 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:48.146501 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bf4774669-vqxdg" event={"ID":"7d64e645-f005-4b2d-8949-50043bf093f3","Type":"ContainerStarted","Data":"e007e581f93ae98f657a6fd6452c2428d13501c66c940bb9a6a04c37ea8bec6b"} Jul 02 23:08:51.160830 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:51.160792 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bf4774669-vqxdg" event={"ID":"7d64e645-f005-4b2d-8949-50043bf093f3","Type":"ContainerStarted","Data":"53ff947a41212270ca28542aa86435c72fa8aa29da8dce270bca35c47379a9c8"} Jul 02 23:08:51.178009 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:51.177948 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5bf4774669-vqxdg" podStartSLOduration=1.435847728 podStartE2EDuration="4.177933227s" totalImagesPullingTime="2.742085499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:08:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:08:51.177217899 +0000 UTC m=+95.200036030" watchObservedRunningTime="2026-07-02 23:08:51.177933227 +0000 UTC m=+95.200751339" Jul 02 23:08:52.352259 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:52.352221 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bf4774669-vqxdg"] Jul 02 23:08:57.440405 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:57.440359 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:08:58.068174 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:08:58.068142 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-drz87" Jul 02 23:09:00.702826 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:00.702797 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vblxw_952de78d-e0fb-4c00-88d6-ca117114fd67/serve-healthcheck-canary/0.log" Jul 02 23:09:18.189049 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.188980 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5bf4774669-vqxdg" podUID="7d64e645-f005-4b2d-8949-50043bf093f3" containerName="console" containerID="cri-o://53ff947a41212270ca28542aa86435c72fa8aa29da8dce270bca35c47379a9c8" gracePeriod=15 Jul 02 23:09:18.429223 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.429195 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:09:18.602921 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.602857 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d64e645-f005-4b2d-8949-50043bf093f3-console-oauth-config\") pod \"7d64e645-f005-4b2d-8949-50043bf093f3\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " Jul 02 23:09:18.603118 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.602964 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-trusted-ca-bundle\") pod \"7d64e645-f005-4b2d-8949-50043bf093f3\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " Jul 02 23:09:18.603118 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.602998 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-service-ca\") pod \"7d64e645-f005-4b2d-8949-50043bf093f3\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " Jul 02 23:09:18.603118 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.603028 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d64e645-f005-4b2d-8949-50043bf093f3-console-serving-cert\") pod \"7d64e645-f005-4b2d-8949-50043bf093f3\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " Jul 02 23:09:18.603118 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.603064 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-oauth-serving-cert\") pod \"7d64e645-f005-4b2d-8949-50043bf093f3\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " Jul 02 23:09:18.603118 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.603097 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-console-config\") pod \"7d64e645-f005-4b2d-8949-50043bf093f3\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " Jul 02 23:09:18.603369 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.603121 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fk644\" (UniqueName: \"kubernetes.io/projected/7d64e645-f005-4b2d-8949-50043bf093f3-kube-api-access-fk644\") pod \"7d64e645-f005-4b2d-8949-50043bf093f3\" (UID: \"7d64e645-f005-4b2d-8949-50043bf093f3\") " Jul 02 23:09:18.603458 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.603423 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-service-ca" (OuterVolumeSpecName: "service-ca") pod "7d64e645-f005-4b2d-8949-50043bf093f3" (UID: "7d64e645-f005-4b2d-8949-50043bf093f3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:09:18.603458 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.603440 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7d64e645-f005-4b2d-8949-50043bf093f3" (UID: "7d64e645-f005-4b2d-8949-50043bf093f3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:09:18.603583 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.603490 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-console-config" (OuterVolumeSpecName: "console-config") pod "7d64e645-f005-4b2d-8949-50043bf093f3" (UID: "7d64e645-f005-4b2d-8949-50043bf093f3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:09:18.603583 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.603501 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7d64e645-f005-4b2d-8949-50043bf093f3" (UID: "7d64e645-f005-4b2d-8949-50043bf093f3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:09:18.605273 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.605244 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d64e645-f005-4b2d-8949-50043bf093f3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7d64e645-f005-4b2d-8949-50043bf093f3" (UID: "7d64e645-f005-4b2d-8949-50043bf093f3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:09:18.605400 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.605382 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d64e645-f005-4b2d-8949-50043bf093f3-kube-api-access-fk644" (OuterVolumeSpecName: "kube-api-access-fk644") pod "7d64e645-f005-4b2d-8949-50043bf093f3" (UID: "7d64e645-f005-4b2d-8949-50043bf093f3"). InnerVolumeSpecName "kube-api-access-fk644". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:09:18.605463 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.605389 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d64e645-f005-4b2d-8949-50043bf093f3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7d64e645-f005-4b2d-8949-50043bf093f3" (UID: "7d64e645-f005-4b2d-8949-50043bf093f3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:09:18.703726 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.703680 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d64e645-f005-4b2d-8949-50043bf093f3-console-oauth-config\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:09:18.703726 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.703705 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-trusted-ca-bundle\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:09:18.703726 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.703715 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-service-ca\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:09:18.703726 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.703724 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d64e645-f005-4b2d-8949-50043bf093f3-console-serving-cert\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:09:18.703726 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.703733 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-oauth-serving-cert\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:09:18.703726 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.703742 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d64e645-f005-4b2d-8949-50043bf093f3-console-config\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:09:18.704213 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:18.703750 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fk644\" (UniqueName: \"kubernetes.io/projected/7d64e645-f005-4b2d-8949-50043bf093f3-kube-api-access-fk644\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:09:19.266331 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:19.266297 2583 generic.go:356] "Generic (PLEG): container finished" podID="7d64e645-f005-4b2d-8949-50043bf093f3" containerID="53ff947a41212270ca28542aa86435c72fa8aa29da8dce270bca35c47379a9c8" exitCode=0 Jul 02 23:09:19.266774 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:19.266368 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bf4774669-vqxdg" Jul 02 23:09:19.266774 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:19.266393 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bf4774669-vqxdg" event={"ID":"7d64e645-f005-4b2d-8949-50043bf093f3","Type":"ContainerDied","Data":"53ff947a41212270ca28542aa86435c72fa8aa29da8dce270bca35c47379a9c8"} Jul 02 23:09:19.266774 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:19.266466 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bf4774669-vqxdg" event={"ID":"7d64e645-f005-4b2d-8949-50043bf093f3","Type":"ContainerDied","Data":"e007e581f93ae98f657a6fd6452c2428d13501c66c940bb9a6a04c37ea8bec6b"} Jul 02 23:09:19.266774 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:19.266487 2583 scope.go:117] "RemoveContainer" containerID="53ff947a41212270ca28542aa86435c72fa8aa29da8dce270bca35c47379a9c8" Jul 02 23:09:19.275120 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:19.275101 2583 scope.go:117] "RemoveContainer" containerID="53ff947a41212270ca28542aa86435c72fa8aa29da8dce270bca35c47379a9c8" Jul 02 23:09:19.275412 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:09:19.275392 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53ff947a41212270ca28542aa86435c72fa8aa29da8dce270bca35c47379a9c8\": container with ID starting with 53ff947a41212270ca28542aa86435c72fa8aa29da8dce270bca35c47379a9c8 not found: ID does not exist" containerID="53ff947a41212270ca28542aa86435c72fa8aa29da8dce270bca35c47379a9c8" Jul 02 23:09:19.275458 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:19.275432 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53ff947a41212270ca28542aa86435c72fa8aa29da8dce270bca35c47379a9c8"} err="failed to get container status \"53ff947a41212270ca28542aa86435c72fa8aa29da8dce270bca35c47379a9c8\": rpc error: code = NotFound desc = could not find container \"53ff947a41212270ca28542aa86435c72fa8aa29da8dce270bca35c47379a9c8\": container with ID starting with 53ff947a41212270ca28542aa86435c72fa8aa29da8dce270bca35c47379a9c8 not found: ID does not exist" Jul 02 23:09:19.286745 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:19.286719 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bf4774669-vqxdg"] Jul 02 23:09:19.290133 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:19.290107 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5bf4774669-vqxdg"] Jul 02 23:09:20.533165 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:09:20.533131 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d64e645-f005-4b2d-8949-50043bf093f3" path="/var/lib/kubelet/pods/7d64e645-f005-4b2d-8949-50043bf093f3/volumes" Jul 02 23:10:07.647774 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.647700 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6994f9f6b4-gv4kh"] Jul 02 23:10:07.648190 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.647943 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d64e645-f005-4b2d-8949-50043bf093f3" containerName="console" Jul 02 23:10:07.648190 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.647955 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d64e645-f005-4b2d-8949-50043bf093f3" containerName="console" Jul 02 23:10:07.648190 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.648022 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d64e645-f005-4b2d-8949-50043bf093f3" containerName="console" Jul 02 23:10:07.650498 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.650475 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.653026 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.652998 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 02 23:10:07.653818 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.653793 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 02 23:10:07.653954 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.653935 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vmc2w\"" Jul 02 23:10:07.654024 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.653962 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 02 23:10:07.654230 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.654214 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 02 23:10:07.654294 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.654250 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 02 23:10:07.654294 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.654272 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 02 23:10:07.654877 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.654858 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 02 23:10:07.657853 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.657741 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 02 23:10:07.663406 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.663385 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6994f9f6b4-gv4kh"] Jul 02 23:10:07.740628 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.740587 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-service-ca\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.740628 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.740622 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-console-config\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.740845 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.740645 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-console-serving-cert\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.740845 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.740708 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-console-oauth-config\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.740845 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.740784 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-oauth-serving-cert\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.740845 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.740811 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj5d7\" (UniqueName: \"kubernetes.io/projected/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-kube-api-access-sj5d7\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.741012 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.740854 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-trusted-ca-bundle\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.842152 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.842107 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-oauth-serving-cert\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.842152 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.842159 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj5d7\" (UniqueName: \"kubernetes.io/projected/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-kube-api-access-sj5d7\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.842420 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.842208 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-trusted-ca-bundle\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.842420 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.842248 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-service-ca\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.842420 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.842374 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-console-config\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.842578 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.842427 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-console-serving-cert\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.842578 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.842457 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-console-oauth-config\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.843016 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.842989 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-service-ca\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.843150 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.842989 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-oauth-serving-cert\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.843212 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.843151 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-console-config\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.843331 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.843205 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-trusted-ca-bundle\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.844986 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.844962 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-console-oauth-config\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.845181 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.845160 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-console-serving-cert\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.849960 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.849940 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj5d7\" (UniqueName: \"kubernetes.io/projected/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-kube-api-access-sj5d7\") pod \"console-6994f9f6b4-gv4kh\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:07.960831 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:07.960742 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:08.088602 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:08.088564 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6994f9f6b4-gv4kh"] Jul 02 23:10:08.091817 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:10:08.091783 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd47c0cc9_9975_43eb_b74f_a5512e1dfed7.slice/crio-70b1c4f974475594367af5e93d5fa58442dae5b3d6165edcd903d218e7f3aa64 WatchSource:0}: Error finding container 70b1c4f974475594367af5e93d5fa58442dae5b3d6165edcd903d218e7f3aa64: Status 404 returned error can't find the container with id 70b1c4f974475594367af5e93d5fa58442dae5b3d6165edcd903d218e7f3aa64 Jul 02 23:10:08.426663 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:08.426627 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6994f9f6b4-gv4kh" event={"ID":"d47c0cc9-9975-43eb-b74f-a5512e1dfed7","Type":"ContainerStarted","Data":"7d913c87cb391b346c8dd3a167808b85324f61f503b21b19a916aa992495cb84"} Jul 02 23:10:08.426834 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:08.426677 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6994f9f6b4-gv4kh" event={"ID":"d47c0cc9-9975-43eb-b74f-a5512e1dfed7","Type":"ContainerStarted","Data":"70b1c4f974475594367af5e93d5fa58442dae5b3d6165edcd903d218e7f3aa64"} Jul 02 23:10:08.443514 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:08.443453 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6994f9f6b4-gv4kh" podStartSLOduration=1.4434365 podStartE2EDuration="1.4434365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:08.443081506 +0000 UTC m=+172.465899621" watchObservedRunningTime="2026-07-02 23:10:08.4434365 +0000 UTC m=+172.466254618" Jul 02 23:10:17.961667 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:17.961630 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:17.962103 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:17.961692 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:17.966645 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:17.966621 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:10:18.466567 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:10:18.466539 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:11:16.665584 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:16.665538 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-22qj6"] Jul 02 23:11:16.668727 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:16.668704 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-22qj6" Jul 02 23:11:16.671008 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:16.670986 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 02 23:11:16.671800 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:16.671784 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 02 23:11:16.671946 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:16.671868 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-fs7rz\"" Jul 02 23:11:16.677874 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:16.677851 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-22qj6"] Jul 02 23:11:16.753623 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:16.753589 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b80409a-bd27-4875-96ad-d2da7a583577-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-22qj6\" (UID: \"4b80409a-bd27-4875-96ad-d2da7a583577\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-22qj6" Jul 02 23:11:16.753805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:16.753643 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfrj2\" (UniqueName: \"kubernetes.io/projected/4b80409a-bd27-4875-96ad-d2da7a583577-kube-api-access-rfrj2\") pod \"cert-manager-cainjector-559cdd8c87-22qj6\" (UID: \"4b80409a-bd27-4875-96ad-d2da7a583577\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-22qj6" Jul 02 23:11:16.854849 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:16.854811 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b80409a-bd27-4875-96ad-d2da7a583577-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-22qj6\" (UID: \"4b80409a-bd27-4875-96ad-d2da7a583577\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-22qj6" Jul 02 23:11:16.855000 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:16.854859 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfrj2\" (UniqueName: \"kubernetes.io/projected/4b80409a-bd27-4875-96ad-d2da7a583577-kube-api-access-rfrj2\") pod \"cert-manager-cainjector-559cdd8c87-22qj6\" (UID: \"4b80409a-bd27-4875-96ad-d2da7a583577\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-22qj6" Jul 02 23:11:16.865409 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:16.865381 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b80409a-bd27-4875-96ad-d2da7a583577-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-22qj6\" (UID: \"4b80409a-bd27-4875-96ad-d2da7a583577\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-22qj6" Jul 02 23:11:16.866035 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:16.866014 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfrj2\" (UniqueName: \"kubernetes.io/projected/4b80409a-bd27-4875-96ad-d2da7a583577-kube-api-access-rfrj2\") pod \"cert-manager-cainjector-559cdd8c87-22qj6\" (UID: \"4b80409a-bd27-4875-96ad-d2da7a583577\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-22qj6" Jul 02 23:11:16.978523 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:16.978423 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-22qj6" Jul 02 23:11:17.106710 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:17.106674 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-22qj6"] Jul 02 23:11:17.109928 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:11:17.109876 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b80409a_bd27_4875_96ad_d2da7a583577.slice/crio-db7c50d6973f11d95d0c9ac34bf157478ef8af9e9b7b8cdefea6f9dd43ceb24e WatchSource:0}: Error finding container db7c50d6973f11d95d0c9ac34bf157478ef8af9e9b7b8cdefea6f9dd43ceb24e: Status 404 returned error can't find the container with id db7c50d6973f11d95d0c9ac34bf157478ef8af9e9b7b8cdefea6f9dd43ceb24e Jul 02 23:11:17.655823 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:17.655784 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-22qj6" event={"ID":"4b80409a-bd27-4875-96ad-d2da7a583577","Type":"ContainerStarted","Data":"db7c50d6973f11d95d0c9ac34bf157478ef8af9e9b7b8cdefea6f9dd43ceb24e"} Jul 02 23:11:20.669818 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:20.669775 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-22qj6" event={"ID":"4b80409a-bd27-4875-96ad-d2da7a583577","Type":"ContainerStarted","Data":"93cf06a7154985841b8f5b98fc05fb62c33317a3401d87148a2c0d97c3554a55"} Jul 02 23:11:20.685611 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:20.685555 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-22qj6" podStartSLOduration=1.541653353 podStartE2EDuration="4.685540265s" totalImagesPullingTime="3.143886912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:20.683826124 +0000 UTC m=+244.706644236" watchObservedRunningTime="2026-07-02 23:11:20.685540265 +0000 UTC m=+244.708358377" Jul 02 23:11:29.515844 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:29.515808 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbcgl"] Jul 02 23:11:29.518816 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:29.518797 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbcgl" Jul 02 23:11:29.521049 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:29.521030 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 02 23:11:29.521148 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:29.521084 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:11:29.521742 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:29.521725 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-lgtf7\"" Jul 02 23:11:29.526468 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:29.526438 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbcgl"] Jul 02 23:11:29.656708 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:29.656668 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f1b26aa-e4ea-47ce-b953-0ecef03d1b74-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rbcgl\" (UID: \"7f1b26aa-e4ea-47ce-b953-0ecef03d1b74\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbcgl" Jul 02 23:11:29.656921 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:29.656722 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59f4r\" (UniqueName: \"kubernetes.io/projected/7f1b26aa-e4ea-47ce-b953-0ecef03d1b74-kube-api-access-59f4r\") pod \"openshift-lws-operator-fd8ccff4c-rbcgl\" (UID: \"7f1b26aa-e4ea-47ce-b953-0ecef03d1b74\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbcgl" Jul 02 23:11:29.758082 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:29.758037 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f1b26aa-e4ea-47ce-b953-0ecef03d1b74-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rbcgl\" (UID: \"7f1b26aa-e4ea-47ce-b953-0ecef03d1b74\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbcgl" Jul 02 23:11:29.758260 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:29.758091 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59f4r\" (UniqueName: \"kubernetes.io/projected/7f1b26aa-e4ea-47ce-b953-0ecef03d1b74-kube-api-access-59f4r\") pod \"openshift-lws-operator-fd8ccff4c-rbcgl\" (UID: \"7f1b26aa-e4ea-47ce-b953-0ecef03d1b74\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbcgl" Jul 02 23:11:29.758432 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:29.758410 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f1b26aa-e4ea-47ce-b953-0ecef03d1b74-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rbcgl\" (UID: \"7f1b26aa-e4ea-47ce-b953-0ecef03d1b74\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbcgl" Jul 02 23:11:29.766213 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:29.766145 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59f4r\" (UniqueName: \"kubernetes.io/projected/7f1b26aa-e4ea-47ce-b953-0ecef03d1b74-kube-api-access-59f4r\") pod \"openshift-lws-operator-fd8ccff4c-rbcgl\" (UID: \"7f1b26aa-e4ea-47ce-b953-0ecef03d1b74\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbcgl" Jul 02 23:11:29.828357 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:29.828320 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbcgl" Jul 02 23:11:29.962684 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:29.962651 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbcgl"] Jul 02 23:11:29.966151 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:11:29.966121 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f1b26aa_e4ea_47ce_b953_0ecef03d1b74.slice/crio-e724cf00a442a889687cf26b8f0d9df8a67ae5b31c30711d20f186fb7d7c4127 WatchSource:0}: Error finding container e724cf00a442a889687cf26b8f0d9df8a67ae5b31c30711d20f186fb7d7c4127: Status 404 returned error can't find the container with id e724cf00a442a889687cf26b8f0d9df8a67ae5b31c30711d20f186fb7d7c4127 Jul 02 23:11:30.707435 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:30.707399 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbcgl" event={"ID":"7f1b26aa-e4ea-47ce-b953-0ecef03d1b74","Type":"ContainerStarted","Data":"e724cf00a442a889687cf26b8f0d9df8a67ae5b31c30711d20f186fb7d7c4127"} Jul 02 23:11:38.737805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:38.737763 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbcgl" event={"ID":"7f1b26aa-e4ea-47ce-b953-0ecef03d1b74","Type":"ContainerStarted","Data":"ab660d77df07eed8286d97ded14058883dc662b3bf0f5f672ccd24a10f292dd7"} Jul 02 23:11:38.753845 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:38.753793 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbcgl" podStartSLOduration=1.070992797 podStartE2EDuration="9.753774267s" totalImagesPullingTime="8.68278147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:38.753117184 +0000 UTC m=+262.775935299" watchObservedRunningTime="2026-07-02 23:11:38.753774267 +0000 UTC m=+262.776592377" Jul 02 23:11:54.009268 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.009233 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc"] Jul 02 23:11:54.048254 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.048222 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc"] Jul 02 23:11:54.048405 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.048340 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" Jul 02 23:11:54.052115 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.052090 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mfznl\"" Jul 02 23:11:54.052261 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.052133 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 02 23:11:54.052261 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.052164 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 02 23:11:54.052445 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.052431 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 02 23:11:54.055084 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.055064 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 02 23:11:54.137051 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.137010 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7a14a32a-7506-4b68-b7b5-64dc31561af1-webhook-cert\") pod \"opendatahub-operator-controller-manager-fb7747bfc-bh6tc\" (UID: \"7a14a32a-7506-4b68-b7b5-64dc31561af1\") " pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" Jul 02 23:11:54.137224 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.137075 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zmqg\" (UniqueName: \"kubernetes.io/projected/7a14a32a-7506-4b68-b7b5-64dc31561af1-kube-api-access-9zmqg\") pod \"opendatahub-operator-controller-manager-fb7747bfc-bh6tc\" (UID: \"7a14a32a-7506-4b68-b7b5-64dc31561af1\") " pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" Jul 02 23:11:54.137224 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.137155 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7a14a32a-7506-4b68-b7b5-64dc31561af1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fb7747bfc-bh6tc\" (UID: \"7a14a32a-7506-4b68-b7b5-64dc31561af1\") " pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" Jul 02 23:11:54.137224 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.137196 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a14a32a-7506-4b68-b7b5-64dc31561af1-tmp\") pod \"opendatahub-operator-controller-manager-fb7747bfc-bh6tc\" (UID: \"7a14a32a-7506-4b68-b7b5-64dc31561af1\") " pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" Jul 02 23:11:54.137224 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.137222 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7a14a32a-7506-4b68-b7b5-64dc31561af1-manifests\") pod \"opendatahub-operator-controller-manager-fb7747bfc-bh6tc\" (UID: \"7a14a32a-7506-4b68-b7b5-64dc31561af1\") " pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" Jul 02 23:11:54.238044 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.237995 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zmqg\" (UniqueName: \"kubernetes.io/projected/7a14a32a-7506-4b68-b7b5-64dc31561af1-kube-api-access-9zmqg\") pod \"opendatahub-operator-controller-manager-fb7747bfc-bh6tc\" (UID: \"7a14a32a-7506-4b68-b7b5-64dc31561af1\") " pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" Jul 02 23:11:54.238044 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.238052 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7a14a32a-7506-4b68-b7b5-64dc31561af1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fb7747bfc-bh6tc\" (UID: \"7a14a32a-7506-4b68-b7b5-64dc31561af1\") " pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" Jul 02 23:11:54.238299 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.238080 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a14a32a-7506-4b68-b7b5-64dc31561af1-tmp\") pod \"opendatahub-operator-controller-manager-fb7747bfc-bh6tc\" (UID: \"7a14a32a-7506-4b68-b7b5-64dc31561af1\") " pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" Jul 02 23:11:54.238299 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.238104 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7a14a32a-7506-4b68-b7b5-64dc31561af1-manifests\") pod \"opendatahub-operator-controller-manager-fb7747bfc-bh6tc\" (UID: \"7a14a32a-7506-4b68-b7b5-64dc31561af1\") " pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" Jul 02 23:11:54.238299 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.238149 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7a14a32a-7506-4b68-b7b5-64dc31561af1-webhook-cert\") pod \"opendatahub-operator-controller-manager-fb7747bfc-bh6tc\" (UID: \"7a14a32a-7506-4b68-b7b5-64dc31561af1\") " pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" Jul 02 23:11:54.238680 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.238650 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7a14a32a-7506-4b68-b7b5-64dc31561af1-manifests\") pod \"opendatahub-operator-controller-manager-fb7747bfc-bh6tc\" (UID: \"7a14a32a-7506-4b68-b7b5-64dc31561af1\") " pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" Jul 02 23:11:54.240643 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.240621 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a14a32a-7506-4b68-b7b5-64dc31561af1-tmp\") pod \"opendatahub-operator-controller-manager-fb7747bfc-bh6tc\" (UID: \"7a14a32a-7506-4b68-b7b5-64dc31561af1\") " pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" Jul 02 23:11:54.240926 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.240883 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7a14a32a-7506-4b68-b7b5-64dc31561af1-webhook-cert\") pod \"opendatahub-operator-controller-manager-fb7747bfc-bh6tc\" (UID: \"7a14a32a-7506-4b68-b7b5-64dc31561af1\") " pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" Jul 02 23:11:54.241027 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.241008 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7a14a32a-7506-4b68-b7b5-64dc31561af1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fb7747bfc-bh6tc\" (UID: \"7a14a32a-7506-4b68-b7b5-64dc31561af1\") " pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" Jul 02 23:11:54.246701 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.246680 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zmqg\" (UniqueName: \"kubernetes.io/projected/7a14a32a-7506-4b68-b7b5-64dc31561af1-kube-api-access-9zmqg\") pod \"opendatahub-operator-controller-manager-fb7747bfc-bh6tc\" (UID: \"7a14a32a-7506-4b68-b7b5-64dc31561af1\") " pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" Jul 02 23:11:54.359685 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.359646 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" Jul 02 23:11:54.505155 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.505129 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc"] Jul 02 23:11:54.508022 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:11:54.507999 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a14a32a_7506_4b68_b7b5_64dc31561af1.slice/crio-75be99eec1ec29a648e7c6057cc40ea70d68b9062b6c0b6d4d7e1f66a9dec7ba WatchSource:0}: Error finding container 75be99eec1ec29a648e7c6057cc40ea70d68b9062b6c0b6d4d7e1f66a9dec7ba: Status 404 returned error can't find the container with id 75be99eec1ec29a648e7c6057cc40ea70d68b9062b6c0b6d4d7e1f66a9dec7ba Jul 02 23:11:54.790394 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:54.790308 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" event={"ID":"7a14a32a-7506-4b68-b7b5-64dc31561af1","Type":"ContainerStarted","Data":"75be99eec1ec29a648e7c6057cc40ea70d68b9062b6c0b6d4d7e1f66a9dec7ba"} Jul 02 23:11:57.803801 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:57.803763 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" event={"ID":"7a14a32a-7506-4b68-b7b5-64dc31561af1","Type":"ContainerStarted","Data":"69bd130ed656915e64f21c87097846e0ec75896bb1f6adcc5e3820fd722f2312"} Jul 02 23:11:58.808396 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:58.808359 2583 generic.go:356] "Generic (PLEG): container finished" podID="7a14a32a-7506-4b68-b7b5-64dc31561af1" containerID="69bd130ed656915e64f21c87097846e0ec75896bb1f6adcc5e3820fd722f2312" exitCode=0 Jul 02 23:11:58.808792 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:58.808446 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" event={"ID":"7a14a32a-7506-4b68-b7b5-64dc31561af1","Type":"ContainerDied","Data":"69bd130ed656915e64f21c87097846e0ec75896bb1f6adcc5e3820fd722f2312"} Jul 02 23:11:59.813766 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:59.813722 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" event={"ID":"7a14a32a-7506-4b68-b7b5-64dc31561af1","Type":"ContainerStarted","Data":"eac76f62ce6879604e2996804234ce5cdad71b2b697d0cf7217c18e3bbc46519"} Jul 02 23:11:59.814227 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:59.813851 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" Jul 02 23:11:59.834795 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:11:59.834730 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" podStartSLOduration=2.4209356890000002 podStartE2EDuration="6.834713636s" totalImagesPullingTime="3.378160356s" totalInitContainerRuntime="1.035617591s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:59.833525329 +0000 UTC m=+283.856343441" watchObservedRunningTime="2026-07-02 23:11:59.834713636 +0000 UTC m=+283.857531827" Jul 02 23:12:10.819850 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:10.819809 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-fb7747bfc-bh6tc" Jul 02 23:12:14.541000 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:14.540969 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-n48md"] Jul 02 23:12:14.545383 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:14.545361 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-n48md" Jul 02 23:12:14.548283 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:14.548262 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 02 23:12:14.558547 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:14.558513 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-n48md"] Jul 02 23:12:14.595647 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:14.595609 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7gsd\" (UniqueName: \"kubernetes.io/projected/c43b5a76-ece9-47b6-bab2-c44debab52e2-kube-api-access-f7gsd\") pod \"model-serving-api-54d89bd5fc-n48md\" (UID: \"c43b5a76-ece9-47b6-bab2-c44debab52e2\") " pod="opendatahub/model-serving-api-54d89bd5fc-n48md" Jul 02 23:12:14.595825 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:14.595663 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c43b5a76-ece9-47b6-bab2-c44debab52e2-tls-certs\") pod \"model-serving-api-54d89bd5fc-n48md\" (UID: \"c43b5a76-ece9-47b6-bab2-c44debab52e2\") " pod="opendatahub/model-serving-api-54d89bd5fc-n48md" Jul 02 23:12:14.696982 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:14.696943 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7gsd\" (UniqueName: \"kubernetes.io/projected/c43b5a76-ece9-47b6-bab2-c44debab52e2-kube-api-access-f7gsd\") pod \"model-serving-api-54d89bd5fc-n48md\" (UID: \"c43b5a76-ece9-47b6-bab2-c44debab52e2\") " pod="opendatahub/model-serving-api-54d89bd5fc-n48md" Jul 02 23:12:14.697157 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:14.696990 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c43b5a76-ece9-47b6-bab2-c44debab52e2-tls-certs\") pod \"model-serving-api-54d89bd5fc-n48md\" (UID: \"c43b5a76-ece9-47b6-bab2-c44debab52e2\") " pod="opendatahub/model-serving-api-54d89bd5fc-n48md" Jul 02 23:12:14.699601 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:14.699582 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c43b5a76-ece9-47b6-bab2-c44debab52e2-tls-certs\") pod \"model-serving-api-54d89bd5fc-n48md\" (UID: \"c43b5a76-ece9-47b6-bab2-c44debab52e2\") " pod="opendatahub/model-serving-api-54d89bd5fc-n48md" Jul 02 23:12:14.704868 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:14.704843 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7gsd\" (UniqueName: \"kubernetes.io/projected/c43b5a76-ece9-47b6-bab2-c44debab52e2-kube-api-access-f7gsd\") pod \"model-serving-api-54d89bd5fc-n48md\" (UID: \"c43b5a76-ece9-47b6-bab2-c44debab52e2\") " pod="opendatahub/model-serving-api-54d89bd5fc-n48md" Jul 02 23:12:14.856084 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:14.856042 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-n48md" Jul 02 23:12:14.987044 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:14.987011 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-n48md"] Jul 02 23:12:14.990881 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:12:14.990849 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc43b5a76_ece9_47b6_bab2_c44debab52e2.slice/crio-7586c4ba43951db62d3c595332219c803d879edfa84d1af3019c880e7f8dbe98 WatchSource:0}: Error finding container 7586c4ba43951db62d3c595332219c803d879edfa84d1af3019c880e7f8dbe98: Status 404 returned error can't find the container with id 7586c4ba43951db62d3c595332219c803d879edfa84d1af3019c880e7f8dbe98 Jul 02 23:12:15.873750 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:15.873709 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-n48md" event={"ID":"c43b5a76-ece9-47b6-bab2-c44debab52e2","Type":"ContainerStarted","Data":"7586c4ba43951db62d3c595332219c803d879edfa84d1af3019c880e7f8dbe98"} Jul 02 23:12:16.371610 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:16.371516 2583 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 23:12:16.879100 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:16.879068 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-n48md" event={"ID":"c43b5a76-ece9-47b6-bab2-c44debab52e2","Type":"ContainerStarted","Data":"10fddfd3be17e2072998346c534dceb79fd98bd3e5df18b11c643b5852516429"} Jul 02 23:12:16.879505 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:16.879191 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-n48md" Jul 02 23:12:16.895081 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:16.895025 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-n48md" podStartSLOduration=1.116678271 podStartE2EDuration="2.895003525s" totalImagesPullingTime="1.778325254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:12:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:12:16.893273395 +0000 UTC m=+300.916091509" watchObservedRunningTime="2026-07-02 23:12:16.895003525 +0000 UTC m=+300.917821641" Jul 02 23:12:19.127440 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.127402 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-55fc97dffd-5wh2r"] Jul 02 23:12:19.130671 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.130642 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55fc97dffd-5wh2r" Jul 02 23:12:19.133128 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.133091 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 02 23:12:19.133244 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.133150 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 02 23:12:19.133244 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.133193 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 02 23:12:19.133244 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.133208 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 02 23:12:19.134448 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.134424 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-pv9d7\"" Jul 02 23:12:19.141572 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.141549 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55fc97dffd-5wh2r"] Jul 02 23:12:19.229472 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.229435 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfb1e9db-f7e1-4221-8dc8-41fd8f51f117-tls-certs\") pod \"kube-auth-proxy-55fc97dffd-5wh2r\" (UID: \"dfb1e9db-f7e1-4221-8dc8-41fd8f51f117\") " pod="openshift-ingress/kube-auth-proxy-55fc97dffd-5wh2r" Jul 02 23:12:19.229472 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.229482 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfb1e9db-f7e1-4221-8dc8-41fd8f51f117-tmp\") pod \"kube-auth-proxy-55fc97dffd-5wh2r\" (UID: \"dfb1e9db-f7e1-4221-8dc8-41fd8f51f117\") " pod="openshift-ingress/kube-auth-proxy-55fc97dffd-5wh2r" Jul 02 23:12:19.229700 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.229527 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qndz6\" (UniqueName: \"kubernetes.io/projected/dfb1e9db-f7e1-4221-8dc8-41fd8f51f117-kube-api-access-qndz6\") pod \"kube-auth-proxy-55fc97dffd-5wh2r\" (UID: \"dfb1e9db-f7e1-4221-8dc8-41fd8f51f117\") " pod="openshift-ingress/kube-auth-proxy-55fc97dffd-5wh2r" Jul 02 23:12:19.330780 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.330732 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qndz6\" (UniqueName: \"kubernetes.io/projected/dfb1e9db-f7e1-4221-8dc8-41fd8f51f117-kube-api-access-qndz6\") pod \"kube-auth-proxy-55fc97dffd-5wh2r\" (UID: \"dfb1e9db-f7e1-4221-8dc8-41fd8f51f117\") " pod="openshift-ingress/kube-auth-proxy-55fc97dffd-5wh2r" Jul 02 23:12:19.331013 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.330820 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfb1e9db-f7e1-4221-8dc8-41fd8f51f117-tls-certs\") pod \"kube-auth-proxy-55fc97dffd-5wh2r\" (UID: \"dfb1e9db-f7e1-4221-8dc8-41fd8f51f117\") " pod="openshift-ingress/kube-auth-proxy-55fc97dffd-5wh2r" Jul 02 23:12:19.331013 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.330861 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfb1e9db-f7e1-4221-8dc8-41fd8f51f117-tmp\") pod \"kube-auth-proxy-55fc97dffd-5wh2r\" (UID: \"dfb1e9db-f7e1-4221-8dc8-41fd8f51f117\") " pod="openshift-ingress/kube-auth-proxy-55fc97dffd-5wh2r" Jul 02 23:12:19.333340 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.333299 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfb1e9db-f7e1-4221-8dc8-41fd8f51f117-tmp\") pod \"kube-auth-proxy-55fc97dffd-5wh2r\" (UID: \"dfb1e9db-f7e1-4221-8dc8-41fd8f51f117\") " pod="openshift-ingress/kube-auth-proxy-55fc97dffd-5wh2r" Jul 02 23:12:19.333519 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.333502 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfb1e9db-f7e1-4221-8dc8-41fd8f51f117-tls-certs\") pod \"kube-auth-proxy-55fc97dffd-5wh2r\" (UID: \"dfb1e9db-f7e1-4221-8dc8-41fd8f51f117\") " pod="openshift-ingress/kube-auth-proxy-55fc97dffd-5wh2r" Jul 02 23:12:19.338686 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.338658 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qndz6\" (UniqueName: \"kubernetes.io/projected/dfb1e9db-f7e1-4221-8dc8-41fd8f51f117-kube-api-access-qndz6\") pod \"kube-auth-proxy-55fc97dffd-5wh2r\" (UID: \"dfb1e9db-f7e1-4221-8dc8-41fd8f51f117\") " pod="openshift-ingress/kube-auth-proxy-55fc97dffd-5wh2r" Jul 02 23:12:19.447308 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.447216 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55fc97dffd-5wh2r" Jul 02 23:12:19.583790 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.583756 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55fc97dffd-5wh2r"] Jul 02 23:12:19.590144 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:12:19.590109 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfb1e9db_f7e1_4221_8dc8_41fd8f51f117.slice/crio-ae9958f1590af883b64ca2ccf35993ec93a6a70c274dcea6f00c1afe7d305ae7 WatchSource:0}: Error finding container ae9958f1590af883b64ca2ccf35993ec93a6a70c274dcea6f00c1afe7d305ae7: Status 404 returned error can't find the container with id ae9958f1590af883b64ca2ccf35993ec93a6a70c274dcea6f00c1afe7d305ae7 Jul 02 23:12:19.591932 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.591882 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:12:19.891610 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:19.891568 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55fc97dffd-5wh2r" event={"ID":"dfb1e9db-f7e1-4221-8dc8-41fd8f51f117","Type":"ContainerStarted","Data":"ae9958f1590af883b64ca2ccf35993ec93a6a70c274dcea6f00c1afe7d305ae7"} Jul 02 23:12:20.342275 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.342243 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-ppq94"] Jul 02 23:12:20.345292 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.345273 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.347618 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.347584 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 02 23:12:20.347766 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.347647 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 02 23:12:20.347766 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.347695 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-tzqch\"" Jul 02 23:12:20.347766 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.347650 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 02 23:12:20.347766 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.347650 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 02 23:12:20.359557 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.359530 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-ppq94"] Jul 02 23:12:20.438560 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.438519 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-token\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.438749 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.438565 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.438749 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.438596 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brrk8\" (UniqueName: \"kubernetes.io/projected/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-kube-api-access-brrk8\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.438749 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.438625 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-local-certs\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.438749 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.438661 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.438985 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.438756 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.438985 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.438796 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-cacerts\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.540008 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.539975 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.540199 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.540019 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-cacerts\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.540199 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.540089 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-token\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.540199 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.540112 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.540199 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.540136 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brrk8\" (UniqueName: \"kubernetes.io/projected/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-kube-api-access-brrk8\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.540199 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.540164 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-local-certs\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.540199 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.540200 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.540801 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.540686 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.542986 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.542936 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-local-certs\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.543634 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.543606 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.543754 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.543606 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-cacerts\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.543754 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.543614 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.554847 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.554814 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-token\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.555067 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.555044 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brrk8\" (UniqueName: \"kubernetes.io/projected/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-kube-api-access-brrk8\") pod \"istiod-openshift-gateway-7574459656-ppq94\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.654775 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.654685 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:20.816550 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.816522 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-ppq94"] Jul 02 23:12:20.819586 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:12:20.819548 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7ff3a4d_65ed_4cf5_8c10_caffb51ee7b4.slice/crio-5004cfc389bde0528b8c36a9bcae59ee443971b7e7c6449e4498117872369974 WatchSource:0}: Error finding container 5004cfc389bde0528b8c36a9bcae59ee443971b7e7c6449e4498117872369974: Status 404 returned error can't find the container with id 5004cfc389bde0528b8c36a9bcae59ee443971b7e7c6449e4498117872369974 Jul 02 23:12:20.898153 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:20.898107 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" event={"ID":"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4","Type":"ContainerStarted","Data":"5004cfc389bde0528b8c36a9bcae59ee443971b7e7c6449e4498117872369974"} Jul 02 23:12:23.416030 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:23.415984 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:12:23.416327 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:23.416072 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:12:23.915137 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:23.915102 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55fc97dffd-5wh2r" event={"ID":"dfb1e9db-f7e1-4221-8dc8-41fd8f51f117","Type":"ContainerStarted","Data":"dc0ff27c1effc203ff673f600c9ef35f1195c2c4f8e38f6a039837881daa9e2f"} Jul 02 23:12:23.916756 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:23.916726 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" event={"ID":"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4","Type":"ContainerStarted","Data":"60c9c6fa3c0fa167a293646d8dfc4cfeeece307e266a1ee89ffbcc0efe5db359"} Jul 02 23:12:23.916873 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:23.916803 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:23.956716 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:23.956657 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-55fc97dffd-5wh2r" podStartSLOduration=1.209445104 podStartE2EDuration="4.95663541s" totalImagesPullingTime="3.747190306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:12:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:12:23.954513371 +0000 UTC m=+307.977331483" watchObservedRunningTime="2026-07-02 23:12:23.95663541 +0000 UTC m=+307.979453525" Jul 02 23:12:24.018036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.017985 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" podStartSLOduration=1.424293445 podStartE2EDuration="4.01796695s" totalImagesPullingTime="2.593673505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:12:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:12:24.015145214 +0000 UTC m=+308.037963327" watchObservedRunningTime="2026-07-02 23:12:24.01796695 +0000 UTC m=+308.040785063" Jul 02 23:12:24.313872 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.313833 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5nccz"] Jul 02 23:12:24.319519 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.318181 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lgpbn"] Jul 02 23:12:24.319718 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.319587 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5nccz" Jul 02 23:12:24.323638 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.321695 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lgpbn" Jul 02 23:12:24.324423 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.324400 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 02 23:12:24.324924 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.324906 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-dpbxk\"" Jul 02 23:12:24.325289 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.325265 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-m8zh7\"" Jul 02 23:12:24.325583 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.325563 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 02 23:12:24.328410 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.328389 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5nccz"] Jul 02 23:12:24.342374 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.342350 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lgpbn"] Jul 02 23:12:24.372758 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.372722 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/26539bc2-ca5b-4dae-a116-5475c9ff79b5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lgpbn\" (UID: \"26539bc2-ca5b-4dae-a116-5475c9ff79b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lgpbn" Jul 02 23:12:24.372982 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.372762 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbszw\" (UniqueName: \"kubernetes.io/projected/26539bc2-ca5b-4dae-a116-5475c9ff79b5-kube-api-access-gbszw\") pod \"llmisvc-controller-manager-777c6bd7d5-lgpbn\" (UID: \"26539bc2-ca5b-4dae-a116-5475c9ff79b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lgpbn" Jul 02 23:12:24.372982 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.372830 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3dc2f7e3-a8b3-4280-a2f1-828b24282c70-cert\") pod \"kserve-controller-manager-d6f4b65b8-5nccz\" (UID: \"3dc2f7e3-a8b3-4280-a2f1-828b24282c70\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5nccz" Jul 02 23:12:24.372982 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.372919 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zlvb\" (UniqueName: \"kubernetes.io/projected/3dc2f7e3-a8b3-4280-a2f1-828b24282c70-kube-api-access-9zlvb\") pod \"kserve-controller-manager-d6f4b65b8-5nccz\" (UID: \"3dc2f7e3-a8b3-4280-a2f1-828b24282c70\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5nccz" Jul 02 23:12:24.473287 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.473252 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zlvb\" (UniqueName: \"kubernetes.io/projected/3dc2f7e3-a8b3-4280-a2f1-828b24282c70-kube-api-access-9zlvb\") pod \"kserve-controller-manager-d6f4b65b8-5nccz\" (UID: \"3dc2f7e3-a8b3-4280-a2f1-828b24282c70\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5nccz" Jul 02 23:12:24.473688 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.473311 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/26539bc2-ca5b-4dae-a116-5475c9ff79b5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lgpbn\" (UID: \"26539bc2-ca5b-4dae-a116-5475c9ff79b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lgpbn" Jul 02 23:12:24.473688 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.473333 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbszw\" (UniqueName: \"kubernetes.io/projected/26539bc2-ca5b-4dae-a116-5475c9ff79b5-kube-api-access-gbszw\") pod \"llmisvc-controller-manager-777c6bd7d5-lgpbn\" (UID: \"26539bc2-ca5b-4dae-a116-5475c9ff79b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lgpbn" Jul 02 23:12:24.473688 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.473369 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3dc2f7e3-a8b3-4280-a2f1-828b24282c70-cert\") pod \"kserve-controller-manager-d6f4b65b8-5nccz\" (UID: \"3dc2f7e3-a8b3-4280-a2f1-828b24282c70\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5nccz" Jul 02 23:12:24.475910 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.475863 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/26539bc2-ca5b-4dae-a116-5475c9ff79b5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lgpbn\" (UID: \"26539bc2-ca5b-4dae-a116-5475c9ff79b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lgpbn" Jul 02 23:12:24.476056 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.476032 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3dc2f7e3-a8b3-4280-a2f1-828b24282c70-cert\") pod \"kserve-controller-manager-d6f4b65b8-5nccz\" (UID: \"3dc2f7e3-a8b3-4280-a2f1-828b24282c70\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5nccz" Jul 02 23:12:24.484493 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.484461 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zlvb\" (UniqueName: \"kubernetes.io/projected/3dc2f7e3-a8b3-4280-a2f1-828b24282c70-kube-api-access-9zlvb\") pod \"kserve-controller-manager-d6f4b65b8-5nccz\" (UID: \"3dc2f7e3-a8b3-4280-a2f1-828b24282c70\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5nccz" Jul 02 23:12:24.484663 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.484639 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbszw\" (UniqueName: \"kubernetes.io/projected/26539bc2-ca5b-4dae-a116-5475c9ff79b5-kube-api-access-gbszw\") pod \"llmisvc-controller-manager-777c6bd7d5-lgpbn\" (UID: \"26539bc2-ca5b-4dae-a116-5475c9ff79b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lgpbn" Jul 02 23:12:24.637061 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.636968 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5nccz" Jul 02 23:12:24.644265 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.644241 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lgpbn" Jul 02 23:12:24.794628 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.794600 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5nccz"] Jul 02 23:12:24.797141 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:12:24.797112 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dc2f7e3_a8b3_4280_a2f1_828b24282c70.slice/crio-e7620b5dd6609c97a15a8c232c85864ff130a39943cdecc941c4157664dee2ed WatchSource:0}: Error finding container e7620b5dd6609c97a15a8c232c85864ff130a39943cdecc941c4157664dee2ed: Status 404 returned error can't find the container with id e7620b5dd6609c97a15a8c232c85864ff130a39943cdecc941c4157664dee2ed Jul 02 23:12:24.820708 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.820565 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lgpbn"] Jul 02 23:12:24.823100 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:12:24.823070 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod26539bc2_ca5b_4dae_a116_5475c9ff79b5.slice/crio-433faa275d8ff788dafa4a77644b37a4b18f104f34bc9fe00f6be105c60c75e4 WatchSource:0}: Error finding container 433faa275d8ff788dafa4a77644b37a4b18f104f34bc9fe00f6be105c60c75e4: Status 404 returned error can't find the container with id 433faa275d8ff788dafa4a77644b37a4b18f104f34bc9fe00f6be105c60c75e4 Jul 02 23:12:24.921881 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.921781 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5nccz" event={"ID":"3dc2f7e3-a8b3-4280-a2f1-828b24282c70","Type":"ContainerStarted","Data":"e7620b5dd6609c97a15a8c232c85864ff130a39943cdecc941c4157664dee2ed"} Jul 02 23:12:24.922953 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.922926 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lgpbn" event={"ID":"26539bc2-ca5b-4dae-a116-5475c9ff79b5","Type":"ContainerStarted","Data":"433faa275d8ff788dafa4a77644b37a4b18f104f34bc9fe00f6be105c60c75e4"} Jul 02 23:12:24.924691 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.924663 2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-ppq94 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 02 23:12:24.924773 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:24.924710 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" podUID="e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 23:12:25.384276 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.384237 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8"] Jul 02 23:12:25.387953 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.387931 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.391054 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.391027 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-6jn58\"" Jul 02 23:12:25.402139 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.402113 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8"] Jul 02 23:12:25.482902 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.482847 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.483459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.482943 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.483459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.482983 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.483459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.483024 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjmk6\" (UniqueName: \"kubernetes.io/projected/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-kube-api-access-tjmk6\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.483459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.483060 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.483459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.483125 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.483459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.483155 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.483459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.483194 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.483459 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.483217 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.584668 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.584602 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.584668 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.584668 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.584953 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.584700 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.584953 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.584733 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjmk6\" (UniqueName: \"kubernetes.io/projected/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-kube-api-access-tjmk6\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.584953 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.584767 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.584953 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.584813 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.584953 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.584832 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.584953 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.584856 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.584953 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.584872 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.585302 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.585042 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.585361 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.585346 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.585522 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.585492 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.585983 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.585955 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.586244 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.586193 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.588029 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.587974 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.588194 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.588172 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.598623 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.598592 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.599229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.599204 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjmk6\" (UniqueName: \"kubernetes.io/projected/b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9-kube-api-access-tjmk6\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678m49s8\" (UID: \"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.699373 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.699299 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:25.862862 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:25.862827 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8"] Jul 02 23:12:26.010659 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:12:26.010566 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5cfec4f_0a2b_46e4_9e41_3545eba0b2d9.slice/crio-62b0974c15e49a6abf74cf8a092b90a250161c35e4a2c4953212871e12fb6670 WatchSource:0}: Error finding container 62b0974c15e49a6abf74cf8a092b90a250161c35e4a2c4953212871e12fb6670: Status 404 returned error can't find the container with id 62b0974c15e49a6abf74cf8a092b90a250161c35e4a2c4953212871e12fb6670 Jul 02 23:12:26.586467 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.586419 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-ppq94"] Jul 02 23:12:26.586957 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.586625 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" podUID="e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4" containerName="discovery" containerID="cri-o://60c9c6fa3c0fa167a293646d8dfc4cfeeece307e266a1ee89ffbcc0efe5db359" gracePeriod=30 Jul 02 23:12:26.587351 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.587330 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:26.870920 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.870874 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:26.899561 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.899520 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-kubeconfig\") pod \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " Jul 02 23:12:26.899727 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.899575 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-local-certs\") pod \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " Jul 02 23:12:26.899727 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.899635 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brrk8\" (UniqueName: \"kubernetes.io/projected/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-kube-api-access-brrk8\") pod \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " Jul 02 23:12:26.899727 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.899677 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-cacerts\") pod \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " Jul 02 23:12:26.899727 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.899715 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-csr-dns-cert\") pod \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " Jul 02 23:12:26.899959 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.899733 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-token\") pod \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " Jul 02 23:12:26.899959 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.899750 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-csr-ca-configmap\") pod \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\" (UID: \"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4\") " Jul 02 23:12:26.900268 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.900212 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4" (UID: "e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:12:26.902825 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.902777 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-local-certs" (OuterVolumeSpecName: "local-certs") pod "e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4" (UID: "e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:12:26.902825 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.902783 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4" (UID: "e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:12:26.903073 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.903049 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-token" (OuterVolumeSpecName: "istio-token") pod "e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4" (UID: "e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:12:26.903563 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.903307 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-kube-api-access-brrk8" (OuterVolumeSpecName: "kube-api-access-brrk8") pod "e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4" (UID: "e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4"). InnerVolumeSpecName "kube-api-access-brrk8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:12:26.903563 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.903462 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-cacerts" (OuterVolumeSpecName: "cacerts") pod "e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4" (UID: "e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:12:26.903834 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.903806 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4" (UID: "e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:12:26.935565 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.935525 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" event={"ID":"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9","Type":"ContainerStarted","Data":"62b0974c15e49a6abf74cf8a092b90a250161c35e4a2c4953212871e12fb6670"} Jul 02 23:12:26.937378 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.937343 2583 generic.go:356] "Generic (PLEG): container finished" podID="e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4" containerID="60c9c6fa3c0fa167a293646d8dfc4cfeeece307e266a1ee89ffbcc0efe5db359" exitCode=0 Jul 02 23:12:26.937594 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.937396 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" Jul 02 23:12:26.937773 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.937415 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" event={"ID":"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4","Type":"ContainerDied","Data":"60c9c6fa3c0fa167a293646d8dfc4cfeeece307e266a1ee89ffbcc0efe5db359"} Jul 02 23:12:26.938009 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.937962 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ppq94" event={"ID":"e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4","Type":"ContainerDied","Data":"5004cfc389bde0528b8c36a9bcae59ee443971b7e7c6449e4498117872369974"} Jul 02 23:12:26.938106 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.938083 2583 scope.go:117] "RemoveContainer" containerID="60c9c6fa3c0fa167a293646d8dfc4cfeeece307e266a1ee89ffbcc0efe5db359" Jul 02 23:12:26.971811 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.971780 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-ppq94"] Jul 02 23:12:26.976945 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:26.976914 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-ppq94"] Jul 02 23:12:27.001112 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:27.001075 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brrk8\" (UniqueName: \"kubernetes.io/projected/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-kube-api-access-brrk8\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:12:27.001112 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:27.001108 2583 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-cacerts\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:12:27.001112 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:27.001121 2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-csr-dns-cert\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:12:27.001396 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:27.001131 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-token\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:12:27.001396 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:27.001140 2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-csr-ca-configmap\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:12:27.001396 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:27.001150 2583 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-istio-kubeconfig\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:12:27.001396 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:27.001163 2583 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4-local-certs\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:12:27.888470 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:27.888436 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-n48md" Jul 02 23:12:28.170687 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:28.170316 2583 scope.go:117] "RemoveContainer" containerID="60c9c6fa3c0fa167a293646d8dfc4cfeeece307e266a1ee89ffbcc0efe5db359" Jul 02 23:12:28.170806 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:12:28.170784 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60c9c6fa3c0fa167a293646d8dfc4cfeeece307e266a1ee89ffbcc0efe5db359\": container with ID starting with 60c9c6fa3c0fa167a293646d8dfc4cfeeece307e266a1ee89ffbcc0efe5db359 not found: ID does not exist" containerID="60c9c6fa3c0fa167a293646d8dfc4cfeeece307e266a1ee89ffbcc0efe5db359" Jul 02 23:12:28.170855 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:28.170821 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60c9c6fa3c0fa167a293646d8dfc4cfeeece307e266a1ee89ffbcc0efe5db359"} err="failed to get container status \"60c9c6fa3c0fa167a293646d8dfc4cfeeece307e266a1ee89ffbcc0efe5db359\": rpc error: code = NotFound desc = could not find container \"60c9c6fa3c0fa167a293646d8dfc4cfeeece307e266a1ee89ffbcc0efe5db359\": container with ID starting with 60c9c6fa3c0fa167a293646d8dfc4cfeeece307e266a1ee89ffbcc0efe5db359 not found: ID does not exist" Jul 02 23:12:28.533167 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:28.533090 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4" path="/var/lib/kubelet/pods/e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4/volumes" Jul 02 23:12:28.949684 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:28.949623 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lgpbn" event={"ID":"26539bc2-ca5b-4dae-a116-5475c9ff79b5","Type":"ContainerStarted","Data":"d460187d9e04ebe15b9ccd783b0ab04e3ac20986e7c7aab1571578ef909cac17"} Jul 02 23:12:28.950200 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:28.949775 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lgpbn" Jul 02 23:12:28.952464 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:28.952431 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5nccz" event={"ID":"3dc2f7e3-a8b3-4280-a2f1-828b24282c70","Type":"ContainerStarted","Data":"81b9cec7d5970516f3623fb7d14d95c501e69f957493e120519cae6e594caa07"} Jul 02 23:12:28.952683 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:28.952584 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5nccz" Jul 02 23:12:28.971923 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:28.971846 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lgpbn" podStartSLOduration=1.612627928 podStartE2EDuration="4.971820544s" totalImagesPullingTime="3.359192616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:12:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:12:28.971729724 +0000 UTC m=+312.994547840" watchObservedRunningTime="2026-07-02 23:12:28.971820544 +0000 UTC m=+312.994638659" Jul 02 23:12:29.707230 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:29.707187 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:12:29.707308 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:29.707274 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:12:29.707308 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:29.707303 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:12:29.958725 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:29.958628 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" event={"ID":"b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9","Type":"ContainerStarted","Data":"bce42fc6ef8113062e75694e40d7a09b558007dcff533c81157059fa63a91893"} Jul 02 23:12:29.995038 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:29.994976 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" podStartSLOduration=1.301446271 podStartE2EDuration="4.994953542s" totalImagesPullingTime="3.693507271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:12:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:12:29.993329318 +0000 UTC m=+314.016147432" watchObservedRunningTime="2026-07-02 23:12:29.994953542 +0000 UTC m=+314.017771656" Jul 02 23:12:29.996371 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:29.996333 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5nccz" podStartSLOduration=2.615107756 podStartE2EDuration="5.996318817s" totalImagesPullingTime="3.381211061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:12:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:12:29.006145036 +0000 UTC m=+313.028963162" watchObservedRunningTime="2026-07-02 23:12:29.996318817 +0000 UTC m=+314.019136932" Jul 02 23:12:30.699632 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:30.699590 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:30.704237 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:30.704209 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:30.962988 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:30.962902 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:30.964120 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:30.964103 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678m49s8" Jul 02 23:12:59.960874 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:59.960845 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lgpbn" Jul 02 23:12:59.964061 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:12:59.964043 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5nccz" Jul 02 23:13:14.627052 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:14.626970 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kp6hk"] Jul 02 23:13:14.627408 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:14.627264 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4" containerName="discovery" Jul 02 23:13:14.627408 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:14.627275 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4" containerName="discovery" Jul 02 23:13:14.627408 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:14.627322 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7ff3a4d-65ed-4cf5-8c10-caffb51ee7b4" containerName="discovery" Jul 02 23:13:14.629510 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:14.629482 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kp6hk" Jul 02 23:13:14.631867 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:14.631841 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-6gcqt\"" Jul 02 23:13:14.632026 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:14.631961 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 23:13:14.632594 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:14.632573 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 23:13:14.641813 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:14.641791 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kp6hk"] Jul 02 23:13:14.709009 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:14.708969 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27cs6\" (UniqueName: \"kubernetes.io/projected/a86075db-a22b-47b5-bace-b69f6f5a9247-kube-api-access-27cs6\") pod \"kuadrant-operator-catalog-kp6hk\" (UID: \"a86075db-a22b-47b5-bace-b69f6f5a9247\") " pod="kuadrant-system/kuadrant-operator-catalog-kp6hk" Jul 02 23:13:14.809652 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:14.809610 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27cs6\" (UniqueName: \"kubernetes.io/projected/a86075db-a22b-47b5-bace-b69f6f5a9247-kube-api-access-27cs6\") pod \"kuadrant-operator-catalog-kp6hk\" (UID: \"a86075db-a22b-47b5-bace-b69f6f5a9247\") " pod="kuadrant-system/kuadrant-operator-catalog-kp6hk" Jul 02 23:13:14.817709 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:14.817683 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27cs6\" (UniqueName: \"kubernetes.io/projected/a86075db-a22b-47b5-bace-b69f6f5a9247-kube-api-access-27cs6\") pod \"kuadrant-operator-catalog-kp6hk\" (UID: \"a86075db-a22b-47b5-bace-b69f6f5a9247\") " pod="kuadrant-system/kuadrant-operator-catalog-kp6hk" Jul 02 23:13:14.940096 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:14.940014 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kp6hk" Jul 02 23:13:14.972629 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:14.972600 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kp6hk"] Jul 02 23:13:15.075293 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:15.075261 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kp6hk"] Jul 02 23:13:15.078255 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:13:15.078228 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda86075db_a22b_47b5_bace_b69f6f5a9247.slice/crio-37dd5cd2bb5d8f47042c4693883004bb4cfc1f1b54ea6776823f6c0323b89b45 WatchSource:0}: Error finding container 37dd5cd2bb5d8f47042c4693883004bb4cfc1f1b54ea6776823f6c0323b89b45: Status 404 returned error can't find the container with id 37dd5cd2bb5d8f47042c4693883004bb4cfc1f1b54ea6776823f6c0323b89b45 Jul 02 23:13:15.141469 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:15.141434 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kp6hk" event={"ID":"a86075db-a22b-47b5-bace-b69f6f5a9247","Type":"ContainerStarted","Data":"37dd5cd2bb5d8f47042c4693883004bb4cfc1f1b54ea6776823f6c0323b89b45"} Jul 02 23:13:15.185093 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:15.185055 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k5gq5"] Jul 02 23:13:15.188072 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:15.188053 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k5gq5" Jul 02 23:13:15.195872 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:15.195844 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k5gq5"] Jul 02 23:13:15.213160 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:15.213130 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xcbl\" (UniqueName: \"kubernetes.io/projected/0e30f73b-2929-4898-818e-17000a744728-kube-api-access-7xcbl\") pod \"kuadrant-operator-catalog-k5gq5\" (UID: \"0e30f73b-2929-4898-818e-17000a744728\") " pod="kuadrant-system/kuadrant-operator-catalog-k5gq5" Jul 02 23:13:15.314181 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:15.314147 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xcbl\" (UniqueName: \"kubernetes.io/projected/0e30f73b-2929-4898-818e-17000a744728-kube-api-access-7xcbl\") pod \"kuadrant-operator-catalog-k5gq5\" (UID: \"0e30f73b-2929-4898-818e-17000a744728\") " pod="kuadrant-system/kuadrant-operator-catalog-k5gq5" Jul 02 23:13:15.321757 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:15.321730 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xcbl\" (UniqueName: \"kubernetes.io/projected/0e30f73b-2929-4898-818e-17000a744728-kube-api-access-7xcbl\") pod \"kuadrant-operator-catalog-k5gq5\" (UID: \"0e30f73b-2929-4898-818e-17000a744728\") " pod="kuadrant-system/kuadrant-operator-catalog-k5gq5" Jul 02 23:13:15.497665 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:15.497575 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k5gq5" Jul 02 23:13:15.648882 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:15.648847 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k5gq5"] Jul 02 23:13:15.681274 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:13:15.681240 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e30f73b_2929_4898_818e_17000a744728.slice/crio-32dff06ea6d776c2dcf215b9c71007ec429204a6152691f70539d2a298dd2ab4 WatchSource:0}: Error finding container 32dff06ea6d776c2dcf215b9c71007ec429204a6152691f70539d2a298dd2ab4: Status 404 returned error can't find the container with id 32dff06ea6d776c2dcf215b9c71007ec429204a6152691f70539d2a298dd2ab4 Jul 02 23:13:16.148251 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:16.148204 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k5gq5" event={"ID":"0e30f73b-2929-4898-818e-17000a744728","Type":"ContainerStarted","Data":"32dff06ea6d776c2dcf215b9c71007ec429204a6152691f70539d2a298dd2ab4"} Jul 02 23:13:18.157653 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:18.157606 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k5gq5" event={"ID":"0e30f73b-2929-4898-818e-17000a744728","Type":"ContainerStarted","Data":"a1471933429de145044a05ca1a5b7bb3f548a8ae4bba004f30cb252ba638b79b"} Jul 02 23:13:18.159168 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:18.159140 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kp6hk" event={"ID":"a86075db-a22b-47b5-bace-b69f6f5a9247","Type":"ContainerStarted","Data":"f7765482d8ebdf80f30d75ac565db636a5284780f707c7015fdddb69a2815ce0"} Jul 02 23:13:18.159279 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:18.159217 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-kp6hk" podUID="a86075db-a22b-47b5-bace-b69f6f5a9247" containerName="registry-server" containerID="cri-o://f7765482d8ebdf80f30d75ac565db636a5284780f707c7015fdddb69a2815ce0" gracePeriod=2 Jul 02 23:13:18.172795 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:18.172748 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-k5gq5" podStartSLOduration=1.766288049 podStartE2EDuration="3.172725161s" totalImagesPullingTime="1.406437112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:13:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:13:18.171978919 +0000 UTC m=+362.194797032" watchObservedRunningTime="2026-07-02 23:13:18.172725161 +0000 UTC m=+362.195543279" Jul 02 23:13:18.186574 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:18.186522 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-kp6hk" podStartSLOduration=2.179500448 podStartE2EDuration="4.186501594s" totalImagesPullingTime="2.007001146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:13:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:13:18.186209396 +0000 UTC m=+362.209027529" watchObservedRunningTime="2026-07-02 23:13:18.186501594 +0000 UTC m=+362.209319708" Jul 02 23:13:18.400781 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:18.400754 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kp6hk" Jul 02 23:13:18.440024 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:18.439877 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27cs6\" (UniqueName: \"kubernetes.io/projected/a86075db-a22b-47b5-bace-b69f6f5a9247-kube-api-access-27cs6\") pod \"a86075db-a22b-47b5-bace-b69f6f5a9247\" (UID: \"a86075db-a22b-47b5-bace-b69f6f5a9247\") " Jul 02 23:13:18.442188 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:18.442155 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a86075db-a22b-47b5-bace-b69f6f5a9247-kube-api-access-27cs6" (OuterVolumeSpecName: "kube-api-access-27cs6") pod "a86075db-a22b-47b5-bace-b69f6f5a9247" (UID: "a86075db-a22b-47b5-bace-b69f6f5a9247"). InnerVolumeSpecName "kube-api-access-27cs6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:13:18.541399 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:18.541375 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27cs6\" (UniqueName: \"kubernetes.io/projected/a86075db-a22b-47b5-bace-b69f6f5a9247-kube-api-access-27cs6\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:13:19.164221 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:19.164183 2583 generic.go:356] "Generic (PLEG): container finished" podID="a86075db-a22b-47b5-bace-b69f6f5a9247" containerID="f7765482d8ebdf80f30d75ac565db636a5284780f707c7015fdddb69a2815ce0" exitCode=0 Jul 02 23:13:19.164690 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:19.164229 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kp6hk" Jul 02 23:13:19.164690 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:19.164265 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kp6hk" event={"ID":"a86075db-a22b-47b5-bace-b69f6f5a9247","Type":"ContainerDied","Data":"f7765482d8ebdf80f30d75ac565db636a5284780f707c7015fdddb69a2815ce0"} Jul 02 23:13:19.164690 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:19.164329 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kp6hk" event={"ID":"a86075db-a22b-47b5-bace-b69f6f5a9247","Type":"ContainerDied","Data":"37dd5cd2bb5d8f47042c4693883004bb4cfc1f1b54ea6776823f6c0323b89b45"} Jul 02 23:13:19.164690 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:19.164371 2583 scope.go:117] "RemoveContainer" containerID="f7765482d8ebdf80f30d75ac565db636a5284780f707c7015fdddb69a2815ce0" Jul 02 23:13:19.173195 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:19.173177 2583 scope.go:117] "RemoveContainer" containerID="f7765482d8ebdf80f30d75ac565db636a5284780f707c7015fdddb69a2815ce0" Jul 02 23:13:19.173447 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:13:19.173425 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7765482d8ebdf80f30d75ac565db636a5284780f707c7015fdddb69a2815ce0\": container with ID starting with f7765482d8ebdf80f30d75ac565db636a5284780f707c7015fdddb69a2815ce0 not found: ID does not exist" containerID="f7765482d8ebdf80f30d75ac565db636a5284780f707c7015fdddb69a2815ce0" Jul 02 23:13:19.173492 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:19.173458 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7765482d8ebdf80f30d75ac565db636a5284780f707c7015fdddb69a2815ce0"} err="failed to get container status \"f7765482d8ebdf80f30d75ac565db636a5284780f707c7015fdddb69a2815ce0\": rpc error: code = NotFound desc = could not find container \"f7765482d8ebdf80f30d75ac565db636a5284780f707c7015fdddb69a2815ce0\": container with ID starting with f7765482d8ebdf80f30d75ac565db636a5284780f707c7015fdddb69a2815ce0 not found: ID does not exist" Jul 02 23:13:19.179990 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:19.179964 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kp6hk"] Jul 02 23:13:19.183460 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:19.183439 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kp6hk"] Jul 02 23:13:20.531576 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:20.531543 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a86075db-a22b-47b5-bace-b69f6f5a9247" path="/var/lib/kubelet/pods/a86075db-a22b-47b5-bace-b69f6f5a9247/volumes" Jul 02 23:13:25.498816 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:25.498778 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-k5gq5" Jul 02 23:13:25.499259 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:25.498840 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-k5gq5" Jul 02 23:13:25.521842 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:25.521812 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-k5gq5" Jul 02 23:13:26.217190 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:26.217163 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-k5gq5" Jul 02 23:13:40.553008 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:40.552972 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dqv2s"] Jul 02 23:13:40.553477 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:40.553297 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a86075db-a22b-47b5-bace-b69f6f5a9247" containerName="registry-server" Jul 02 23:13:40.553477 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:40.553309 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a86075db-a22b-47b5-bace-b69f6f5a9247" containerName="registry-server" Jul 02 23:13:40.553477 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:40.553390 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a86075db-a22b-47b5-bace-b69f6f5a9247" containerName="registry-server" Jul 02 23:13:40.556278 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:40.556261 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dqv2s" Jul 02 23:13:40.559656 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:40.559637 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-hfhd9\"" Jul 02 23:13:40.575549 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:40.575523 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dqv2s"] Jul 02 23:13:40.630438 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:40.630393 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w79k\" (UniqueName: \"kubernetes.io/projected/592729d8-3f0f-401a-9bc4-6580f56faa57-kube-api-access-6w79k\") pod \"authorino-operator-75cc8d5cb7-dqv2s\" (UID: \"592729d8-3f0f-401a-9bc4-6580f56faa57\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dqv2s" Jul 02 23:13:40.731843 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:40.731800 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w79k\" (UniqueName: \"kubernetes.io/projected/592729d8-3f0f-401a-9bc4-6580f56faa57-kube-api-access-6w79k\") pod \"authorino-operator-75cc8d5cb7-dqv2s\" (UID: \"592729d8-3f0f-401a-9bc4-6580f56faa57\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dqv2s" Jul 02 23:13:40.745480 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:40.745453 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w79k\" (UniqueName: \"kubernetes.io/projected/592729d8-3f0f-401a-9bc4-6580f56faa57-kube-api-access-6w79k\") pod \"authorino-operator-75cc8d5cb7-dqv2s\" (UID: \"592729d8-3f0f-401a-9bc4-6580f56faa57\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dqv2s" Jul 02 23:13:40.865722 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:40.865634 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dqv2s" Jul 02 23:13:41.030532 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:41.030494 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dqv2s"] Jul 02 23:13:41.033762 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:13:41.033734 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod592729d8_3f0f_401a_9bc4_6580f56faa57.slice/crio-c640ea7a5db2938fa8722510a853f4f71d7c73345e995bb6af7161fdaa2e56ea WatchSource:0}: Error finding container c640ea7a5db2938fa8722510a853f4f71d7c73345e995bb6af7161fdaa2e56ea: Status 404 returned error can't find the container with id c640ea7a5db2938fa8722510a853f4f71d7c73345e995bb6af7161fdaa2e56ea Jul 02 23:13:41.257923 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:41.257808 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dqv2s" event={"ID":"592729d8-3f0f-401a-9bc4-6580f56faa57","Type":"ContainerStarted","Data":"c640ea7a5db2938fa8722510a853f4f71d7c73345e995bb6af7161fdaa2e56ea"} Jul 02 23:13:44.273160 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:44.273122 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dqv2s" event={"ID":"592729d8-3f0f-401a-9bc4-6580f56faa57","Type":"ContainerStarted","Data":"585b7a354dd9e2d858bd8c46ccbc5d258dd75e329fc15eeb93db60cb3ef370b3"} Jul 02 23:13:44.273570 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:44.273247 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dqv2s" Jul 02 23:13:44.289208 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:44.289155 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dqv2s" podStartSLOduration=2.086306455 podStartE2EDuration="4.289138205s" totalImagesPullingTime="2.20283175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:13:44.287966978 +0000 UTC m=+388.310785085" watchObservedRunningTime="2026-07-02 23:13:44.289138205 +0000 UTC m=+388.311956319" Jul 02 23:13:44.368066 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:44.368030 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-r9l69"] Jul 02 23:13:44.371331 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:44.371315 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r9l69" Jul 02 23:13:44.373638 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:44.373614 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 02 23:13:44.373793 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:44.373647 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-wdb9b\"" Jul 02 23:13:44.381536 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:44.381512 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-r9l69"] Jul 02 23:13:44.465800 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:44.465767 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2w2d\" (UniqueName: \"kubernetes.io/projected/3dab87d1-a71e-4de9-865c-42cd7ecf5392-kube-api-access-c2w2d\") pod \"dns-operator-controller-manager-864dd78c4c-r9l69\" (UID: \"3dab87d1-a71e-4de9-865c-42cd7ecf5392\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r9l69" Jul 02 23:13:44.566517 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:44.566487 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2w2d\" (UniqueName: \"kubernetes.io/projected/3dab87d1-a71e-4de9-865c-42cd7ecf5392-kube-api-access-c2w2d\") pod \"dns-operator-controller-manager-864dd78c4c-r9l69\" (UID: \"3dab87d1-a71e-4de9-865c-42cd7ecf5392\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r9l69" Jul 02 23:13:44.574119 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:44.574089 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2w2d\" (UniqueName: \"kubernetes.io/projected/3dab87d1-a71e-4de9-865c-42cd7ecf5392-kube-api-access-c2w2d\") pod \"dns-operator-controller-manager-864dd78c4c-r9l69\" (UID: \"3dab87d1-a71e-4de9-865c-42cd7ecf5392\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r9l69" Jul 02 23:13:44.682035 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:44.681987 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r9l69" Jul 02 23:13:44.816480 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:44.816456 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-r9l69"] Jul 02 23:13:44.819159 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:13:44.819128 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dab87d1_a71e_4de9_865c_42cd7ecf5392.slice/crio-953cb21bcd9bee075a494a4949d856bd7c5d6262d88bec5e9bef03efd0f6bfbe WatchSource:0}: Error finding container 953cb21bcd9bee075a494a4949d856bd7c5d6262d88bec5e9bef03efd0f6bfbe: Status 404 returned error can't find the container with id 953cb21bcd9bee075a494a4949d856bd7c5d6262d88bec5e9bef03efd0f6bfbe Jul 02 23:13:45.278509 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:45.278404 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r9l69" event={"ID":"3dab87d1-a71e-4de9-865c-42cd7ecf5392","Type":"ContainerStarted","Data":"953cb21bcd9bee075a494a4949d856bd7c5d6262d88bec5e9bef03efd0f6bfbe"} Jul 02 23:13:47.288204 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:47.288166 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r9l69" event={"ID":"3dab87d1-a71e-4de9-865c-42cd7ecf5392","Type":"ContainerStarted","Data":"5f4cdf55049adb6dc53cec130cb4a5f0ec14d7d4b68150fcc8caa8257ef456d4"} Jul 02 23:13:47.288694 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:47.288250 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r9l69" Jul 02 23:13:47.304227 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:47.304172 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r9l69" podStartSLOduration=1.465143052 podStartE2EDuration="3.30415533s" totalImagesPullingTime="1.839012278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:13:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:13:47.30345844 +0000 UTC m=+391.326276553" watchObservedRunningTime="2026-07-02 23:13:47.30415533 +0000 UTC m=+391.326973443" Jul 02 23:13:47.924974 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:47.924933 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb"] Jul 02 23:13:47.928303 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:47.928279 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" Jul 02 23:13:47.930698 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:47.930680 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-mrhl5\"" Jul 02 23:13:47.940224 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:47.940200 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb"] Jul 02 23:13:48.000042 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:48.000009 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqv2c\" (UniqueName: \"kubernetes.io/projected/c07836f8-3ff5-4b2d-ba5b-3405cad72cfd-kube-api-access-qqv2c\") pod \"limitador-operator-controller-manager-859d7b78bd-pmmrb\" (UID: \"c07836f8-3ff5-4b2d-ba5b-3405cad72cfd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" Jul 02 23:13:48.100953 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:48.100916 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqv2c\" (UniqueName: \"kubernetes.io/projected/c07836f8-3ff5-4b2d-ba5b-3405cad72cfd-kube-api-access-qqv2c\") pod \"limitador-operator-controller-manager-859d7b78bd-pmmrb\" (UID: \"c07836f8-3ff5-4b2d-ba5b-3405cad72cfd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" Jul 02 23:13:48.113780 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:48.113751 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqv2c\" (UniqueName: \"kubernetes.io/projected/c07836f8-3ff5-4b2d-ba5b-3405cad72cfd-kube-api-access-qqv2c\") pod \"limitador-operator-controller-manager-859d7b78bd-pmmrb\" (UID: \"c07836f8-3ff5-4b2d-ba5b-3405cad72cfd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" Jul 02 23:13:48.237874 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:48.237785 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" Jul 02 23:13:48.399618 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:48.399584 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb"] Jul 02 23:13:48.403161 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:13:48.403124 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc07836f8_3ff5_4b2d_ba5b_3405cad72cfd.slice/crio-a80e193f959d7ae0c300065c1b4f8d182eb44f076214256f65495d0f5c7ef248 WatchSource:0}: Error finding container a80e193f959d7ae0c300065c1b4f8d182eb44f076214256f65495d0f5c7ef248: Status 404 returned error can't find the container with id a80e193f959d7ae0c300065c1b4f8d182eb44f076214256f65495d0f5c7ef248 Jul 02 23:13:49.300292 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:49.300256 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" event={"ID":"c07836f8-3ff5-4b2d-ba5b-3405cad72cfd","Type":"ContainerStarted","Data":"a80e193f959d7ae0c300065c1b4f8d182eb44f076214256f65495d0f5c7ef248"} Jul 02 23:13:50.306534 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:50.306498 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" event={"ID":"c07836f8-3ff5-4b2d-ba5b-3405cad72cfd","Type":"ContainerStarted","Data":"b4d737c23d0ba2bd4783f0a4c87b3472cf3c820c78820b998511e4093cb7397f"} Jul 02 23:13:50.307009 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:50.306609 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" Jul 02 23:13:50.322189 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:50.322134 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" podStartSLOduration=1.7291062209999999 podStartE2EDuration="3.32211857s" totalImagesPullingTime="1.593012349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:13:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:13:50.320734802 +0000 UTC m=+394.343552928" watchObservedRunningTime="2026-07-02 23:13:50.32211857 +0000 UTC m=+394.344936682" Jul 02 23:13:52.504410 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:52.504372 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb"] Jul 02 23:13:52.507748 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:52.507733 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:13:52.510042 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:52.510016 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mprfg\"" Jul 02 23:13:52.519816 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:52.519792 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb"] Jul 02 23:13:52.641614 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:52.641579 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6489ecbb-9ede-4908-8179-955ccd872641-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l84xb\" (UID: \"6489ecbb-9ede-4908-8179-955ccd872641\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:13:52.641788 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:52.641641 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwmjr\" (UniqueName: \"kubernetes.io/projected/6489ecbb-9ede-4908-8179-955ccd872641-kube-api-access-fwmjr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l84xb\" (UID: \"6489ecbb-9ede-4908-8179-955ccd872641\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:13:52.742104 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:52.742064 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6489ecbb-9ede-4908-8179-955ccd872641-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l84xb\" (UID: \"6489ecbb-9ede-4908-8179-955ccd872641\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:13:52.742271 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:52.742140 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwmjr\" (UniqueName: \"kubernetes.io/projected/6489ecbb-9ede-4908-8179-955ccd872641-kube-api-access-fwmjr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l84xb\" (UID: \"6489ecbb-9ede-4908-8179-955ccd872641\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:13:52.742441 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:52.742422 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6489ecbb-9ede-4908-8179-955ccd872641-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l84xb\" (UID: \"6489ecbb-9ede-4908-8179-955ccd872641\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:13:52.750567 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:52.750542 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwmjr\" (UniqueName: \"kubernetes.io/projected/6489ecbb-9ede-4908-8179-955ccd872641-kube-api-access-fwmjr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l84xb\" (UID: \"6489ecbb-9ede-4908-8179-955ccd872641\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:13:52.817969 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:52.817935 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:13:52.958148 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:52.958109 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb"] Jul 02 23:13:52.962094 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:13:52.962061 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6489ecbb_9ede_4908_8179_955ccd872641.slice/crio-375ba861495d23d049c0af153509880b578095e5408536cf08d63e0b83effee4 WatchSource:0}: Error finding container 375ba861495d23d049c0af153509880b578095e5408536cf08d63e0b83effee4: Status 404 returned error can't find the container with id 375ba861495d23d049c0af153509880b578095e5408536cf08d63e0b83effee4 Jul 02 23:13:53.320388 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:53.320355 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" event={"ID":"6489ecbb-9ede-4908-8179-955ccd872641","Type":"ContainerStarted","Data":"375ba861495d23d049c0af153509880b578095e5408536cf08d63e0b83effee4"} Jul 02 23:13:55.280952 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:55.280919 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dqv2s" Jul 02 23:13:58.296478 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:58.296443 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r9l69" Jul 02 23:13:59.353361 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:13:59.353327 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" event={"ID":"6489ecbb-9ede-4908-8179-955ccd872641","Type":"ContainerStarted","Data":"4d5f0f27393dfa4fd0c95d2a2ef0c3f5bd344d07b22c36fe3405e2f2adc0d639"} Jul 02 23:14:00.357638 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:00.357598 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:14:00.376769 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:00.376708 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" podStartSLOduration=2.057295206 podStartE2EDuration="8.376688607s" totalImagesPullingTime="6.319393401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:13:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:14:00.375419096 +0000 UTC m=+404.398237210" watchObservedRunningTime="2026-07-02 23:14:00.376688607 +0000 UTC m=+404.399506721" Jul 02 23:14:01.312616 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:01.312581 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" Jul 02 23:14:03.479343 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:03.479309 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6994f9f6b4-gv4kh"] Jul 02 23:14:11.365370 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:11.365336 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:14:13.155724 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.155690 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb"] Jul 02 23:14:13.156437 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.155958 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" podUID="6489ecbb-9ede-4908-8179-955ccd872641" containerName="manager" containerID="cri-o://4d5f0f27393dfa4fd0c95d2a2ef0c3f5bd344d07b22c36fe3405e2f2adc0d639" gracePeriod=2 Jul 02 23:14:13.163793 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.163757 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb"] Jul 02 23:14:13.176380 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:14:13.176334 2583 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"6489ecbb-9ede-4908-8179-955ccd872641\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-l84xb\": pods \"kuadrant-operator-controller-manager-8684c9ff49-l84xb\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:14:13.181026 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.180951 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb"] Jul 02 23:14:13.181301 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.181240 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" podUID="c07836f8-3ff5-4b2d-ba5b-3405cad72cfd" containerName="manager" containerID="cri-o://b4d737c23d0ba2bd4783f0a4c87b3472cf3c820c78820b998511e4093cb7397f" gracePeriod=2 Jul 02 23:14:13.183537 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.183510 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55"] Jul 02 23:14:13.183953 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.183936 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6489ecbb-9ede-4908-8179-955ccd872641" containerName="manager" Jul 02 23:14:13.183953 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.183954 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6489ecbb-9ede-4908-8179-955ccd872641" containerName="manager" Jul 02 23:14:13.184123 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.184031 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6489ecbb-9ede-4908-8179-955ccd872641" containerName="manager" Jul 02 23:14:13.187497 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.187475 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55" Jul 02 23:14:13.192624 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.192599 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb"] Jul 02 23:14:13.202523 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.202491 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55"] Jul 02 23:14:13.204525 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.204497 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pcqnn"] Jul 02 23:14:13.204864 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.204847 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c07836f8-3ff5-4b2d-ba5b-3405cad72cfd" containerName="manager" Jul 02 23:14:13.204962 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.204866 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c07836f8-3ff5-4b2d-ba5b-3405cad72cfd" containerName="manager" Jul 02 23:14:13.205016 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.204975 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c07836f8-3ff5-4b2d-ba5b-3405cad72cfd" containerName="manager" Jul 02 23:14:13.206596 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:14:13.206564 2583 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c07836f8-3ff5-4b2d-ba5b-3405cad72cfd\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-pmmrb\": pods \"limitador-operator-controller-manager-859d7b78bd-pmmrb\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" Jul 02 23:14:13.208308 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.208290 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pcqnn" Jul 02 23:14:13.208542 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:14:13.208511 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l84xb\" is forbidden: User \"system:node:ip-10-0-136-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-243.ec2.internal' and this object" podUID="6489ecbb-9ede-4908-8179-955ccd872641" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:14:13.210859 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.210839 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99psj\" (UniqueName: \"kubernetes.io/projected/a0865f5a-b5fd-44f2-affd-c1b7112b4134-kube-api-access-99psj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k2q55\" (UID: \"a0865f5a-b5fd-44f2-affd-c1b7112b4134\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55" Jul 02 23:14:13.211002 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.210986 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0865f5a-b5fd-44f2-affd-c1b7112b4134-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k2q55\" (UID: \"a0865f5a-b5fd-44f2-affd-c1b7112b4134\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55" Jul 02 23:14:13.222528 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.222497 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pcqnn"] Jul 02 23:14:13.239413 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:14:13.239373 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l84xb\" is forbidden: User \"system:node:ip-10-0-136-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-243.ec2.internal' and this object" podUID="6489ecbb-9ede-4908-8179-955ccd872641" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:14:13.241172 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:14:13.241146 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pmmrb\" is forbidden: User \"system:node:ip-10-0-136-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-243.ec2.internal' and this object" podUID="c07836f8-3ff5-4b2d-ba5b-3405cad72cfd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" Jul 02 23:14:13.312039 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.311789 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99psj\" (UniqueName: \"kubernetes.io/projected/a0865f5a-b5fd-44f2-affd-c1b7112b4134-kube-api-access-99psj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k2q55\" (UID: \"a0865f5a-b5fd-44f2-affd-c1b7112b4134\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55" Jul 02 23:14:13.312039 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.311847 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hc9q\" (UniqueName: \"kubernetes.io/projected/0df8f401-e0e6-4bd6-b9a5-96f5e04c92cd-kube-api-access-7hc9q\") pod \"limitador-operator-controller-manager-859d7b78bd-pcqnn\" (UID: \"0df8f401-e0e6-4bd6-b9a5-96f5e04c92cd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pcqnn" Jul 02 23:14:13.312039 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.311911 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0865f5a-b5fd-44f2-affd-c1b7112b4134-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k2q55\" (UID: \"a0865f5a-b5fd-44f2-affd-c1b7112b4134\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55" Jul 02 23:14:13.312614 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.312281 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0865f5a-b5fd-44f2-affd-c1b7112b4134-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k2q55\" (UID: \"a0865f5a-b5fd-44f2-affd-c1b7112b4134\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55" Jul 02 23:14:13.320958 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.320903 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99psj\" (UniqueName: \"kubernetes.io/projected/a0865f5a-b5fd-44f2-affd-c1b7112b4134-kube-api-access-99psj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k2q55\" (UID: \"a0865f5a-b5fd-44f2-affd-c1b7112b4134\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55" Jul 02 23:14:13.413209 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.413092 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hc9q\" (UniqueName: \"kubernetes.io/projected/0df8f401-e0e6-4bd6-b9a5-96f5e04c92cd-kube-api-access-7hc9q\") pod \"limitador-operator-controller-manager-859d7b78bd-pcqnn\" (UID: \"0df8f401-e0e6-4bd6-b9a5-96f5e04c92cd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pcqnn" Jul 02 23:14:13.419210 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.419181 2583 generic.go:356] "Generic (PLEG): container finished" podID="6489ecbb-9ede-4908-8179-955ccd872641" containerID="4d5f0f27393dfa4fd0c95d2a2ef0c3f5bd344d07b22c36fe3405e2f2adc0d639" exitCode=0 Jul 02 23:14:13.419344 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.419252 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="375ba861495d23d049c0af153509880b578095e5408536cf08d63e0b83effee4" Jul 02 23:14:13.420711 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.420693 2583 generic.go:356] "Generic (PLEG): container finished" podID="c07836f8-3ff5-4b2d-ba5b-3405cad72cfd" containerID="b4d737c23d0ba2bd4783f0a4c87b3472cf3c820c78820b998511e4093cb7397f" exitCode=0 Jul 02 23:14:13.421950 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.421929 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hc9q\" (UniqueName: \"kubernetes.io/projected/0df8f401-e0e6-4bd6-b9a5-96f5e04c92cd-kube-api-access-7hc9q\") pod \"limitador-operator-controller-manager-859d7b78bd-pcqnn\" (UID: \"0df8f401-e0e6-4bd6-b9a5-96f5e04c92cd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pcqnn" Jul 02 23:14:13.422485 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.422462 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:14:13.424824 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:14:13.424802 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l84xb\" is forbidden: User \"system:node:ip-10-0-136-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-243.ec2.internal' and this object" podUID="6489ecbb-9ede-4908-8179-955ccd872641" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:14:13.425399 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.425386 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" Jul 02 23:14:13.427007 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:14:13.426988 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pmmrb\" is forbidden: User \"system:node:ip-10-0-136-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-243.ec2.internal' and this object" podUID="c07836f8-3ff5-4b2d-ba5b-3405cad72cfd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" Jul 02 23:14:13.428969 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:14:13.428876 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l84xb\" is forbidden: User \"system:node:ip-10-0-136-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-243.ec2.internal' and this object" podUID="6489ecbb-9ede-4908-8179-955ccd872641" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:14:13.430634 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:14:13.430616 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pmmrb\" is forbidden: User \"system:node:ip-10-0-136-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-243.ec2.internal' and this object" podUID="c07836f8-3ff5-4b2d-ba5b-3405cad72cfd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" Jul 02 23:14:13.513926 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.513859 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqv2c\" (UniqueName: \"kubernetes.io/projected/c07836f8-3ff5-4b2d-ba5b-3405cad72cfd-kube-api-access-qqv2c\") pod \"c07836f8-3ff5-4b2d-ba5b-3405cad72cfd\" (UID: \"c07836f8-3ff5-4b2d-ba5b-3405cad72cfd\") " Jul 02 23:14:13.514125 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.513957 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwmjr\" (UniqueName: \"kubernetes.io/projected/6489ecbb-9ede-4908-8179-955ccd872641-kube-api-access-fwmjr\") pod \"6489ecbb-9ede-4908-8179-955ccd872641\" (UID: \"6489ecbb-9ede-4908-8179-955ccd872641\") " Jul 02 23:14:13.514125 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.514036 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6489ecbb-9ede-4908-8179-955ccd872641-extensions-socket-volume\") pod \"6489ecbb-9ede-4908-8179-955ccd872641\" (UID: \"6489ecbb-9ede-4908-8179-955ccd872641\") " Jul 02 23:14:13.514574 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.514543 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6489ecbb-9ede-4908-8179-955ccd872641-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6489ecbb-9ede-4908-8179-955ccd872641" (UID: "6489ecbb-9ede-4908-8179-955ccd872641"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:14:13.516108 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.516083 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c07836f8-3ff5-4b2d-ba5b-3405cad72cfd-kube-api-access-qqv2c" (OuterVolumeSpecName: "kube-api-access-qqv2c") pod "c07836f8-3ff5-4b2d-ba5b-3405cad72cfd" (UID: "c07836f8-3ff5-4b2d-ba5b-3405cad72cfd"). InnerVolumeSpecName "kube-api-access-qqv2c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:14:13.516194 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.516115 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6489ecbb-9ede-4908-8179-955ccd872641-kube-api-access-fwmjr" (OuterVolumeSpecName: "kube-api-access-fwmjr") pod "6489ecbb-9ede-4908-8179-955ccd872641" (UID: "6489ecbb-9ede-4908-8179-955ccd872641"). InnerVolumeSpecName "kube-api-access-fwmjr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:14:13.606506 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.606469 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55" Jul 02 23:14:13.613732 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.613703 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pcqnn" Jul 02 23:14:13.615151 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.615122 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqv2c\" (UniqueName: \"kubernetes.io/projected/c07836f8-3ff5-4b2d-ba5b-3405cad72cfd-kube-api-access-qqv2c\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:14:13.615241 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.615152 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwmjr\" (UniqueName: \"kubernetes.io/projected/6489ecbb-9ede-4908-8179-955ccd872641-kube-api-access-fwmjr\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:14:13.615241 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.615168 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6489ecbb-9ede-4908-8179-955ccd872641-extensions-socket-volume\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:14:13.767963 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.767940 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55"] Jul 02 23:14:13.769737 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:14:13.769707 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0865f5a_b5fd_44f2_affd_c1b7112b4134.slice/crio-53a9588051ac2e44353cdad33ac6e3c5e4ebd45d0fdc8728f36f790f0142136f WatchSource:0}: Error finding container 53a9588051ac2e44353cdad33ac6e3c5e4ebd45d0fdc8728f36f790f0142136f: Status 404 returned error can't find the container with id 53a9588051ac2e44353cdad33ac6e3c5e4ebd45d0fdc8728f36f790f0142136f Jul 02 23:14:13.782142 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:13.782119 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pcqnn"] Jul 02 23:14:13.787846 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:14:13.787821 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0df8f401_e0e6_4bd6_b9a5_96f5e04c92cd.slice/crio-2f6e8c3ec9bc394ee1a6d464c66fd4799c92f2fda1fadd2dc12b0936af77af18 WatchSource:0}: Error finding container 2f6e8c3ec9bc394ee1a6d464c66fd4799c92f2fda1fadd2dc12b0936af77af18: Status 404 returned error can't find the container with id 2f6e8c3ec9bc394ee1a6d464c66fd4799c92f2fda1fadd2dc12b0936af77af18 Jul 02 23:14:14.426215 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:14.426188 2583 scope.go:117] "RemoveContainer" containerID="b4d737c23d0ba2bd4783f0a4c87b3472cf3c820c78820b998511e4093cb7397f" Jul 02 23:14:14.426663 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:14.426187 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" Jul 02 23:14:14.428154 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:14.428127 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pcqnn" event={"ID":"0df8f401-e0e6-4bd6-b9a5-96f5e04c92cd","Type":"ContainerStarted","Data":"b7c7444088edc5ad1e4a8d7309887822fa48f0cfe427e33ad85e2c344690d5f5"} Jul 02 23:14:14.428286 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:14.428181 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pcqnn" event={"ID":"0df8f401-e0e6-4bd6-b9a5-96f5e04c92cd","Type":"ContainerStarted","Data":"2f6e8c3ec9bc394ee1a6d464c66fd4799c92f2fda1fadd2dc12b0936af77af18"} Jul 02 23:14:14.428286 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:14.428212 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pcqnn" Jul 02 23:14:14.428617 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:14:14.428588 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l84xb\" is forbidden: User \"system:node:ip-10-0-136-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-243.ec2.internal' and this object" podUID="6489ecbb-9ede-4908-8179-955ccd872641" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:14:14.430199 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:14.430171 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:14:14.430199 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:14.430176 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55" event={"ID":"a0865f5a-b5fd-44f2-affd-c1b7112b4134","Type":"ContainerStarted","Data":"8ce126b5ef5f1f567e6b3515afd8b9bd34128037934864db98631be10aed2598"} Jul 02 23:14:14.430346 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:14.430226 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55" event={"ID":"a0865f5a-b5fd-44f2-affd-c1b7112b4134","Type":"ContainerStarted","Data":"53a9588051ac2e44353cdad33ac6e3c5e4ebd45d0fdc8728f36f790f0142136f"} Jul 02 23:14:14.430417 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:14.430351 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55" Jul 02 23:14:14.430417 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:14:14.430397 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pmmrb\" is forbidden: User \"system:node:ip-10-0-136-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-243.ec2.internal' and this object" podUID="c07836f8-3ff5-4b2d-ba5b-3405cad72cfd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" Jul 02 23:14:14.432188 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:14:14.432162 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l84xb\" is forbidden: User \"system:node:ip-10-0-136-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-243.ec2.internal' and this object" podUID="6489ecbb-9ede-4908-8179-955ccd872641" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:14:14.433979 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:14:14.433956 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pmmrb\" is forbidden: User \"system:node:ip-10-0-136-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-243.ec2.internal' and this object" podUID="c07836f8-3ff5-4b2d-ba5b-3405cad72cfd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" Jul 02 23:14:14.452950 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:14.452753 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pcqnn" podStartSLOduration=1.452728559 podStartE2EDuration="1.452728559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:14:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:14:14.452445526 +0000 UTC m=+418.475263639" watchObservedRunningTime="2026-07-02 23:14:14.452728559 +0000 UTC m=+418.475546650" Jul 02 23:14:14.471761 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:14.471712 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55" podStartSLOduration=1.471695858 podStartE2EDuration="1.471695858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:14:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:14:14.470125628 +0000 UTC m=+418.492943741" watchObservedRunningTime="2026-07-02 23:14:14.471695858 +0000 UTC m=+418.494513970" Jul 02 23:14:14.472075 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:14:14.472047 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pmmrb\" is forbidden: User \"system:node:ip-10-0-136-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-243.ec2.internal' and this object" podUID="c07836f8-3ff5-4b2d-ba5b-3405cad72cfd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pmmrb" Jul 02 23:14:14.473603 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:14:14.473580 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l84xb\" is forbidden: User \"system:node:ip-10-0-136-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-243.ec2.internal' and this object" podUID="6489ecbb-9ede-4908-8179-955ccd872641" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84xb" Jul 02 23:14:14.532763 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:14.532727 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6489ecbb-9ede-4908-8179-955ccd872641" path="/var/lib/kubelet/pods/6489ecbb-9ede-4908-8179-955ccd872641/volumes" Jul 02 23:14:14.533071 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:14.533058 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c07836f8-3ff5-4b2d-ba5b-3405cad72cfd" path="/var/lib/kubelet/pods/c07836f8-3ff5-4b2d-ba5b-3405cad72cfd/volumes" Jul 02 23:14:25.437186 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:25.437153 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55" Jul 02 23:14:25.437627 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:25.437234 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pcqnn" Jul 02 23:14:28.500884 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.500812 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6994f9f6b4-gv4kh" podUID="d47c0cc9-9975-43eb-b74f-a5512e1dfed7" containerName="console" containerID="cri-o://7d913c87cb391b346c8dd3a167808b85324f61f503b21b19a916aa992495cb84" gracePeriod=15 Jul 02 23:14:28.736177 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.736152 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:14:28.847104 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.847066 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-trusted-ca-bundle\") pod \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " Jul 02 23:14:28.847290 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.847117 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sj5d7\" (UniqueName: \"kubernetes.io/projected/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-kube-api-access-sj5d7\") pod \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " Jul 02 23:14:28.847290 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.847146 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-service-ca\") pod \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " Jul 02 23:14:28.847290 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.847164 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-oauth-serving-cert\") pod \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " Jul 02 23:14:28.847290 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.847184 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-console-oauth-config\") pod \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " Jul 02 23:14:28.847290 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.847217 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-console-config\") pod \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " Jul 02 23:14:28.847290 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.847268 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-console-serving-cert\") pod \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\" (UID: \"d47c0cc9-9975-43eb-b74f-a5512e1dfed7\") " Jul 02 23:14:28.847641 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.847529 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d47c0cc9-9975-43eb-b74f-a5512e1dfed7" (UID: "d47c0cc9-9975-43eb-b74f-a5512e1dfed7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:14:28.847719 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.847679 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-service-ca" (OuterVolumeSpecName: "service-ca") pod "d47c0cc9-9975-43eb-b74f-a5512e1dfed7" (UID: "d47c0cc9-9975-43eb-b74f-a5512e1dfed7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:14:28.847856 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.847719 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d47c0cc9-9975-43eb-b74f-a5512e1dfed7" (UID: "d47c0cc9-9975-43eb-b74f-a5512e1dfed7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:14:28.847856 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.847736 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-console-config" (OuterVolumeSpecName: "console-config") pod "d47c0cc9-9975-43eb-b74f-a5512e1dfed7" (UID: "d47c0cc9-9975-43eb-b74f-a5512e1dfed7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:14:28.849703 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.849680 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d47c0cc9-9975-43eb-b74f-a5512e1dfed7" (UID: "d47c0cc9-9975-43eb-b74f-a5512e1dfed7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:14:28.850016 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.849997 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d47c0cc9-9975-43eb-b74f-a5512e1dfed7" (UID: "d47c0cc9-9975-43eb-b74f-a5512e1dfed7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:14:28.850069 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.850052 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-kube-api-access-sj5d7" (OuterVolumeSpecName: "kube-api-access-sj5d7") pod "d47c0cc9-9975-43eb-b74f-a5512e1dfed7" (UID: "d47c0cc9-9975-43eb-b74f-a5512e1dfed7"). InnerVolumeSpecName "kube-api-access-sj5d7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:14:28.948672 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.948633 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-trusted-ca-bundle\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:14:28.948672 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.948663 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sj5d7\" (UniqueName: \"kubernetes.io/projected/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-kube-api-access-sj5d7\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:14:28.948672 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.948674 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-service-ca\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:14:28.948672 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.948682 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-oauth-serving-cert\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:14:28.948972 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.948690 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-console-oauth-config\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:14:28.948972 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.948698 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-console-config\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:14:28.948972 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:28.948707 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d47c0cc9-9975-43eb-b74f-a5512e1dfed7-console-serving-cert\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:14:29.496703 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:29.496668 2583 generic.go:356] "Generic (PLEG): container finished" podID="d47c0cc9-9975-43eb-b74f-a5512e1dfed7" containerID="7d913c87cb391b346c8dd3a167808b85324f61f503b21b19a916aa992495cb84" exitCode=0 Jul 02 23:14:29.496921 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:29.496711 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6994f9f6b4-gv4kh" event={"ID":"d47c0cc9-9975-43eb-b74f-a5512e1dfed7","Type":"ContainerDied","Data":"7d913c87cb391b346c8dd3a167808b85324f61f503b21b19a916aa992495cb84"} Jul 02 23:14:29.496921 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:29.496751 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6994f9f6b4-gv4kh" event={"ID":"d47c0cc9-9975-43eb-b74f-a5512e1dfed7","Type":"ContainerDied","Data":"70b1c4f974475594367af5e93d5fa58442dae5b3d6165edcd903d218e7f3aa64"} Jul 02 23:14:29.496921 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:29.496769 2583 scope.go:117] "RemoveContainer" containerID="7d913c87cb391b346c8dd3a167808b85324f61f503b21b19a916aa992495cb84" Jul 02 23:14:29.496921 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:29.496714 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6994f9f6b4-gv4kh" Jul 02 23:14:29.506338 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:29.506191 2583 scope.go:117] "RemoveContainer" containerID="7d913c87cb391b346c8dd3a167808b85324f61f503b21b19a916aa992495cb84" Jul 02 23:14:29.506576 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:14:29.506467 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d913c87cb391b346c8dd3a167808b85324f61f503b21b19a916aa992495cb84\": container with ID starting with 7d913c87cb391b346c8dd3a167808b85324f61f503b21b19a916aa992495cb84 not found: ID does not exist" containerID="7d913c87cb391b346c8dd3a167808b85324f61f503b21b19a916aa992495cb84" Jul 02 23:14:29.506576 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:29.506495 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d913c87cb391b346c8dd3a167808b85324f61f503b21b19a916aa992495cb84"} err="failed to get container status \"7d913c87cb391b346c8dd3a167808b85324f61f503b21b19a916aa992495cb84\": rpc error: code = NotFound desc = could not find container \"7d913c87cb391b346c8dd3a167808b85324f61f503b21b19a916aa992495cb84\": container with ID starting with 7d913c87cb391b346c8dd3a167808b85324f61f503b21b19a916aa992495cb84 not found: ID does not exist" Jul 02 23:14:29.518074 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:29.518041 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6994f9f6b4-gv4kh"] Jul 02 23:14:29.522038 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:29.522013 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6994f9f6b4-gv4kh"] Jul 02 23:14:30.532042 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:30.532006 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d47c0cc9-9975-43eb-b74f-a5512e1dfed7" path="/var/lib/kubelet/pods/d47c0cc9-9975-43eb-b74f-a5512e1dfed7/volumes" Jul 02 23:14:41.632707 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:41.632626 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55"] Jul 02 23:14:41.633167 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:41.632828 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55" podUID="a0865f5a-b5fd-44f2-affd-c1b7112b4134" containerName="manager" containerID="cri-o://8ce126b5ef5f1f567e6b3515afd8b9bd34128037934864db98631be10aed2598" gracePeriod=10 Jul 02 23:14:41.889797 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:41.889724 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55" Jul 02 23:14:41.904796 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:41.904761 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg"] Jul 02 23:14:41.905138 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:41.905124 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d47c0cc9-9975-43eb-b74f-a5512e1dfed7" containerName="console" Jul 02 23:14:41.905138 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:41.905139 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d47c0cc9-9975-43eb-b74f-a5512e1dfed7" containerName="console" Jul 02 23:14:41.905253 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:41.905149 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0865f5a-b5fd-44f2-affd-c1b7112b4134" containerName="manager" Jul 02 23:14:41.905253 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:41.905168 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0865f5a-b5fd-44f2-affd-c1b7112b4134" containerName="manager" Jul 02 23:14:41.905253 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:41.905238 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0865f5a-b5fd-44f2-affd-c1b7112b4134" containerName="manager" Jul 02 23:14:41.905253 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:41.905248 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d47c0cc9-9975-43eb-b74f-a5512e1dfed7" containerName="console" Jul 02 23:14:41.907173 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:41.907157 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg" Jul 02 23:14:41.928308 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:41.928279 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg"] Jul 02 23:14:41.959227 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:41.959198 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0865f5a-b5fd-44f2-affd-c1b7112b4134-extensions-socket-volume\") pod \"a0865f5a-b5fd-44f2-affd-c1b7112b4134\" (UID: \"a0865f5a-b5fd-44f2-affd-c1b7112b4134\") " Jul 02 23:14:41.959402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:41.959245 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99psj\" (UniqueName: \"kubernetes.io/projected/a0865f5a-b5fd-44f2-affd-c1b7112b4134-kube-api-access-99psj\") pod \"a0865f5a-b5fd-44f2-affd-c1b7112b4134\" (UID: \"a0865f5a-b5fd-44f2-affd-c1b7112b4134\") " Jul 02 23:14:41.959547 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:41.959526 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0865f5a-b5fd-44f2-affd-c1b7112b4134-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a0865f5a-b5fd-44f2-affd-c1b7112b4134" (UID: "a0865f5a-b5fd-44f2-affd-c1b7112b4134"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:14:41.961428 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:41.961404 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0865f5a-b5fd-44f2-affd-c1b7112b4134-kube-api-access-99psj" (OuterVolumeSpecName: "kube-api-access-99psj") pod "a0865f5a-b5fd-44f2-affd-c1b7112b4134" (UID: "a0865f5a-b5fd-44f2-affd-c1b7112b4134"). InnerVolumeSpecName "kube-api-access-99psj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:14:42.059758 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.059699 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkzrb\" (UniqueName: \"kubernetes.io/projected/e1642df8-ac92-496a-b122-429dfabdba60-kube-api-access-jkzrb\") pod \"kuadrant-operator-controller-manager-86878c769f-7jjsg\" (UID: \"e1642df8-ac92-496a-b122-429dfabdba60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg" Jul 02 23:14:42.059965 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.059846 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1642df8-ac92-496a-b122-429dfabdba60-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7jjsg\" (UID: \"e1642df8-ac92-496a-b122-429dfabdba60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg" Jul 02 23:14:42.059965 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.059927 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0865f5a-b5fd-44f2-affd-c1b7112b4134-extensions-socket-volume\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:14:42.059965 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.059941 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99psj\" (UniqueName: \"kubernetes.io/projected/a0865f5a-b5fd-44f2-affd-c1b7112b4134-kube-api-access-99psj\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:14:42.161290 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.161204 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1642df8-ac92-496a-b122-429dfabdba60-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7jjsg\" (UID: \"e1642df8-ac92-496a-b122-429dfabdba60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg" Jul 02 23:14:42.161290 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.161250 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkzrb\" (UniqueName: \"kubernetes.io/projected/e1642df8-ac92-496a-b122-429dfabdba60-kube-api-access-jkzrb\") pod \"kuadrant-operator-controller-manager-86878c769f-7jjsg\" (UID: \"e1642df8-ac92-496a-b122-429dfabdba60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg" Jul 02 23:14:42.161603 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.161582 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1642df8-ac92-496a-b122-429dfabdba60-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7jjsg\" (UID: \"e1642df8-ac92-496a-b122-429dfabdba60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg" Jul 02 23:14:42.169192 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.169157 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkzrb\" (UniqueName: \"kubernetes.io/projected/e1642df8-ac92-496a-b122-429dfabdba60-kube-api-access-jkzrb\") pod \"kuadrant-operator-controller-manager-86878c769f-7jjsg\" (UID: \"e1642df8-ac92-496a-b122-429dfabdba60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg" Jul 02 23:14:42.217071 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.217031 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg" Jul 02 23:14:42.363789 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.363757 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg"] Jul 02 23:14:42.367204 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:14:42.367176 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1642df8_ac92_496a_b122_429dfabdba60.slice/crio-dad28abbd24067c0f23ced0f519a0d70a9f9333c0abeaa0d5596fb74cad14e03 WatchSource:0}: Error finding container dad28abbd24067c0f23ced0f519a0d70a9f9333c0abeaa0d5596fb74cad14e03: Status 404 returned error can't find the container with id dad28abbd24067c0f23ced0f519a0d70a9f9333c0abeaa0d5596fb74cad14e03 Jul 02 23:14:42.562487 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.562443 2583 generic.go:356] "Generic (PLEG): container finished" podID="a0865f5a-b5fd-44f2-affd-c1b7112b4134" containerID="8ce126b5ef5f1f567e6b3515afd8b9bd34128037934864db98631be10aed2598" exitCode=0 Jul 02 23:14:42.562666 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.562496 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55" Jul 02 23:14:42.562666 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.562525 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55" event={"ID":"a0865f5a-b5fd-44f2-affd-c1b7112b4134","Type":"ContainerDied","Data":"8ce126b5ef5f1f567e6b3515afd8b9bd34128037934864db98631be10aed2598"} Jul 02 23:14:42.562666 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.562575 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55" event={"ID":"a0865f5a-b5fd-44f2-affd-c1b7112b4134","Type":"ContainerDied","Data":"53a9588051ac2e44353cdad33ac6e3c5e4ebd45d0fdc8728f36f790f0142136f"} Jul 02 23:14:42.562666 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.562605 2583 scope.go:117] "RemoveContainer" containerID="8ce126b5ef5f1f567e6b3515afd8b9bd34128037934864db98631be10aed2598" Jul 02 23:14:42.564375 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.564344 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg" event={"ID":"e1642df8-ac92-496a-b122-429dfabdba60","Type":"ContainerStarted","Data":"2f7e8cb7c6ae67922b1f6c8297cb98a4b75efa63dcdc58b1a72d314ba6bc2fb3"} Jul 02 23:14:42.564700 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.564398 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg" event={"ID":"e1642df8-ac92-496a-b122-429dfabdba60","Type":"ContainerStarted","Data":"dad28abbd24067c0f23ced0f519a0d70a9f9333c0abeaa0d5596fb74cad14e03"} Jul 02 23:14:42.564700 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.564479 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg" Jul 02 23:14:42.571801 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.571779 2583 scope.go:117] "RemoveContainer" containerID="8ce126b5ef5f1f567e6b3515afd8b9bd34128037934864db98631be10aed2598" Jul 02 23:14:42.572084 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:14:42.572066 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ce126b5ef5f1f567e6b3515afd8b9bd34128037934864db98631be10aed2598\": container with ID starting with 8ce126b5ef5f1f567e6b3515afd8b9bd34128037934864db98631be10aed2598 not found: ID does not exist" containerID="8ce126b5ef5f1f567e6b3515afd8b9bd34128037934864db98631be10aed2598" Jul 02 23:14:42.572132 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.572093 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ce126b5ef5f1f567e6b3515afd8b9bd34128037934864db98631be10aed2598"} err="failed to get container status \"8ce126b5ef5f1f567e6b3515afd8b9bd34128037934864db98631be10aed2598\": rpc error: code = NotFound desc = could not find container \"8ce126b5ef5f1f567e6b3515afd8b9bd34128037934864db98631be10aed2598\": container with ID starting with 8ce126b5ef5f1f567e6b3515afd8b9bd34128037934864db98631be10aed2598 not found: ID does not exist" Jul 02 23:14:42.585355 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.585301 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg" podStartSLOduration=1.585282124 podStartE2EDuration="1.585282124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:14:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:14:42.583790149 +0000 UTC m=+446.606608261" watchObservedRunningTime="2026-07-02 23:14:42.585282124 +0000 UTC m=+446.608100238" Jul 02 23:14:42.597035 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.597003 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55"] Jul 02 23:14:42.600644 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:42.600606 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k2q55"] Jul 02 23:14:44.532636 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:44.532602 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0865f5a-b5fd-44f2-affd-c1b7112b4134" path="/var/lib/kubelet/pods/a0865f5a-b5fd-44f2-affd-c1b7112b4134/volumes" Jul 02 23:14:53.573088 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:53.573050 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg" Jul 02 23:14:57.928780 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:57.928745 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54"] Jul 02 23:14:57.933443 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:57.933425 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:57.935805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:57.935780 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-sbfcp\"" Jul 02 23:14:57.941638 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:57.941347 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54"] Jul 02 23:14:58.103313 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.103269 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35aad047-abe8-466e-9e01-ce87f1e2b080-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.103518 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.103328 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35aad047-abe8-466e-9e01-ce87f1e2b080-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.103518 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.103356 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35aad047-abe8-466e-9e01-ce87f1e2b080-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.103518 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.103392 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35aad047-abe8-466e-9e01-ce87f1e2b080-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.103518 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.103421 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35aad047-abe8-466e-9e01-ce87f1e2b080-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.103518 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.103467 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35aad047-abe8-466e-9e01-ce87f1e2b080-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.103518 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.103504 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35aad047-abe8-466e-9e01-ce87f1e2b080-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.103788 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.103529 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35aad047-abe8-466e-9e01-ce87f1e2b080-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.103788 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.103553 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdvs8\" (UniqueName: \"kubernetes.io/projected/35aad047-abe8-466e-9e01-ce87f1e2b080-kube-api-access-kdvs8\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.204971 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.204854 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35aad047-abe8-466e-9e01-ce87f1e2b080-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.204971 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.204905 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35aad047-abe8-466e-9e01-ce87f1e2b080-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.204971 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.204934 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35aad047-abe8-466e-9e01-ce87f1e2b080-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.204971 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.204975 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35aad047-abe8-466e-9e01-ce87f1e2b080-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.205305 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.204996 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35aad047-abe8-466e-9e01-ce87f1e2b080-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.205305 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.205013 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdvs8\" (UniqueName: \"kubernetes.io/projected/35aad047-abe8-466e-9e01-ce87f1e2b080-kube-api-access-kdvs8\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.205305 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.205035 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35aad047-abe8-466e-9e01-ce87f1e2b080-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.205305 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.205075 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35aad047-abe8-466e-9e01-ce87f1e2b080-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.205305 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.205103 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35aad047-abe8-466e-9e01-ce87f1e2b080-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.205558 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.205334 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35aad047-abe8-466e-9e01-ce87f1e2b080-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.205558 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.205427 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35aad047-abe8-466e-9e01-ce87f1e2b080-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.205558 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.205552 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35aad047-abe8-466e-9e01-ce87f1e2b080-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.205720 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.205615 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35aad047-abe8-466e-9e01-ce87f1e2b080-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.205767 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.205744 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35aad047-abe8-466e-9e01-ce87f1e2b080-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.207927 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.207864 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35aad047-abe8-466e-9e01-ce87f1e2b080-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.208054 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.207984 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35aad047-abe8-466e-9e01-ce87f1e2b080-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.213178 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.213156 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35aad047-abe8-466e-9e01-ce87f1e2b080-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.213420 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.213401 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdvs8\" (UniqueName: \"kubernetes.io/projected/35aad047-abe8-466e-9e01-ce87f1e2b080-kube-api-access-kdvs8\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v8h54\" (UID: \"35aad047-abe8-466e-9e01-ce87f1e2b080\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.245368 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.245328 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:58.386021 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.385995 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54"] Jul 02 23:14:58.388456 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:14:58.388427 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35aad047_abe8_466e_9e01_ce87f1e2b080.slice/crio-745c5383ab6f9358b8026a81a5e5327f36a3b4a5430513476863fe63e7f10217 WatchSource:0}: Error finding container 745c5383ab6f9358b8026a81a5e5327f36a3b4a5430513476863fe63e7f10217: Status 404 returned error can't find the container with id 745c5383ab6f9358b8026a81a5e5327f36a3b4a5430513476863fe63e7f10217 Jul 02 23:14:58.390975 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.390941 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:14:58.391070 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.391012 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:14:58.391070 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.391042 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:14:58.639455 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.639421 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" event={"ID":"35aad047-abe8-466e-9e01-ce87f1e2b080","Type":"ContainerStarted","Data":"aa4f7e01e80605439208c807f26c4e16b777a5a4db018e3907f627197a33821e"} Jul 02 23:14:58.639719 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.639484 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" event={"ID":"35aad047-abe8-466e-9e01-ce87f1e2b080","Type":"ContainerStarted","Data":"745c5383ab6f9358b8026a81a5e5327f36a3b4a5430513476863fe63e7f10217"} Jul 02 23:14:58.661741 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:58.661680 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" podStartSLOduration=1.661661632 podStartE2EDuration="1.661661632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:14:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:14:58.660606974 +0000 UTC m=+462.683425086" watchObservedRunningTime="2026-07-02 23:14:58.661661632 +0000 UTC m=+462.684479797" Jul 02 23:14:59.246487 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:59.246439 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:59.251359 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:59.251329 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:59.643631 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:59.643598 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:14:59.644690 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:14:59.644666 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:15:02.921963 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:02.921927 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-wslll"] Jul 02 23:15:02.925326 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:02.925307 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-wslll" Jul 02 23:15:02.927701 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:02.927680 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-b4v5d\"" Jul 02 23:15:02.934482 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:02.934456 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-wslll"] Jul 02 23:15:03.044084 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:03.044046 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2zw8\" (UniqueName: \"kubernetes.io/projected/3d1086f5-e73f-43a9-8697-df2d535dd2ef-kube-api-access-g2zw8\") pod \"authorino-58cd955675-wslll\" (UID: \"3d1086f5-e73f-43a9-8697-df2d535dd2ef\") " pod="kuadrant-system/authorino-58cd955675-wslll" Jul 02 23:15:03.127769 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:03.127725 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-7td9f"] Jul 02 23:15:03.131001 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:03.130979 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7td9f" Jul 02 23:15:03.137035 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:03.137009 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7td9f"] Jul 02 23:15:03.144573 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:03.144538 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2zw8\" (UniqueName: \"kubernetes.io/projected/3d1086f5-e73f-43a9-8697-df2d535dd2ef-kube-api-access-g2zw8\") pod \"authorino-58cd955675-wslll\" (UID: \"3d1086f5-e73f-43a9-8697-df2d535dd2ef\") " pod="kuadrant-system/authorino-58cd955675-wslll" Jul 02 23:15:03.152361 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:03.152328 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2zw8\" (UniqueName: \"kubernetes.io/projected/3d1086f5-e73f-43a9-8697-df2d535dd2ef-kube-api-access-g2zw8\") pod \"authorino-58cd955675-wslll\" (UID: \"3d1086f5-e73f-43a9-8697-df2d535dd2ef\") " pod="kuadrant-system/authorino-58cd955675-wslll" Jul 02 23:15:03.236053 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:03.235971 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-wslll" Jul 02 23:15:03.246958 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:03.246833 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84rhf\" (UniqueName: \"kubernetes.io/projected/fc58071c-cf7d-496e-a9d5-cdbee2076b87-kube-api-access-84rhf\") pod \"authorino-75d556bd77-7td9f\" (UID: \"fc58071c-cf7d-496e-a9d5-cdbee2076b87\") " pod="kuadrant-system/authorino-75d556bd77-7td9f" Jul 02 23:15:03.349778 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:03.349745 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84rhf\" (UniqueName: \"kubernetes.io/projected/fc58071c-cf7d-496e-a9d5-cdbee2076b87-kube-api-access-84rhf\") pod \"authorino-75d556bd77-7td9f\" (UID: \"fc58071c-cf7d-496e-a9d5-cdbee2076b87\") " pod="kuadrant-system/authorino-75d556bd77-7td9f" Jul 02 23:15:03.357396 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:03.357370 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84rhf\" (UniqueName: \"kubernetes.io/projected/fc58071c-cf7d-496e-a9d5-cdbee2076b87-kube-api-access-84rhf\") pod \"authorino-75d556bd77-7td9f\" (UID: \"fc58071c-cf7d-496e-a9d5-cdbee2076b87\") " pod="kuadrant-system/authorino-75d556bd77-7td9f" Jul 02 23:15:03.368180 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:03.368151 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-wslll"] Jul 02 23:15:03.371433 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:15:03.371403 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d1086f5_e73f_43a9_8697_df2d535dd2ef.slice/crio-1de84adc755b9cbebebc7b1c4feab203149aebb23150ac0bd4ffc537ee32e578 WatchSource:0}: Error finding container 1de84adc755b9cbebebc7b1c4feab203149aebb23150ac0bd4ffc537ee32e578: Status 404 returned error can't find the container with id 1de84adc755b9cbebebc7b1c4feab203149aebb23150ac0bd4ffc537ee32e578 Jul 02 23:15:03.441127 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:03.441093 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7td9f" Jul 02 23:15:03.574241 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:03.574206 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7td9f"] Jul 02 23:15:03.576860 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:15:03.576834 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc58071c_cf7d_496e_a9d5_cdbee2076b87.slice/crio-92f82645565a72f8bafa8dfb256e8c4da06cfc168d12b4c221a0e4563fc38119 WatchSource:0}: Error finding container 92f82645565a72f8bafa8dfb256e8c4da06cfc168d12b4c221a0e4563fc38119: Status 404 returned error can't find the container with id 92f82645565a72f8bafa8dfb256e8c4da06cfc168d12b4c221a0e4563fc38119 Jul 02 23:15:03.661971 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:03.661927 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-wslll" event={"ID":"3d1086f5-e73f-43a9-8697-df2d535dd2ef","Type":"ContainerStarted","Data":"1de84adc755b9cbebebc7b1c4feab203149aebb23150ac0bd4ffc537ee32e578"} Jul 02 23:15:03.663037 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:03.663016 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7td9f" event={"ID":"fc58071c-cf7d-496e-a9d5-cdbee2076b87","Type":"ContainerStarted","Data":"92f82645565a72f8bafa8dfb256e8c4da06cfc168d12b4c221a0e4563fc38119"} Jul 02 23:15:07.689076 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:07.689035 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7td9f" event={"ID":"fc58071c-cf7d-496e-a9d5-cdbee2076b87","Type":"ContainerStarted","Data":"ffd50bf0a168c18df0f488c46fedeea203008fdc22c8db065c8da290b6682104"} Jul 02 23:15:07.690383 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:07.690359 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-wslll" event={"ID":"3d1086f5-e73f-43a9-8697-df2d535dd2ef","Type":"ContainerStarted","Data":"222e05f60868da154c6dc1ac0d418816c5ee0cb2d0f2ea33530a2b771a2ba6c4"} Jul 02 23:15:07.704144 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:07.704096 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-7td9f" podStartSLOduration=1.394219188 podStartE2EDuration="4.704080239s" totalImagesPullingTime="3.309861051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:15:07.702109294 +0000 UTC m=+471.724927408" watchObservedRunningTime="2026-07-02 23:15:07.704080239 +0000 UTC m=+471.726898352" Jul 02 23:15:07.717147 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:07.717085 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-wslll" podStartSLOduration=2.212255772 podStartE2EDuration="5.71706342s" totalImagesPullingTime="3.504807648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:15:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:15:07.715078263 +0000 UTC m=+471.737896422" watchObservedRunningTime="2026-07-02 23:15:07.71706342 +0000 UTC m=+471.739881537" Jul 02 23:15:07.741912 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:07.741842 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-wslll"] Jul 02 23:15:09.699199 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:09.699160 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-wslll" podUID="3d1086f5-e73f-43a9-8697-df2d535dd2ef" containerName="authorino" containerID="cri-o://222e05f60868da154c6dc1ac0d418816c5ee0cb2d0f2ea33530a2b771a2ba6c4" gracePeriod=30 Jul 02 23:15:09.957371 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:09.957308 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-wslll" Jul 02 23:15:10.004849 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:10.004812 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2zw8\" (UniqueName: \"kubernetes.io/projected/3d1086f5-e73f-43a9-8697-df2d535dd2ef-kube-api-access-g2zw8\") pod \"3d1086f5-e73f-43a9-8697-df2d535dd2ef\" (UID: \"3d1086f5-e73f-43a9-8697-df2d535dd2ef\") " Jul 02 23:15:10.007035 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:10.007004 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d1086f5-e73f-43a9-8697-df2d535dd2ef-kube-api-access-g2zw8" (OuterVolumeSpecName: "kube-api-access-g2zw8") pod "3d1086f5-e73f-43a9-8697-df2d535dd2ef" (UID: "3d1086f5-e73f-43a9-8697-df2d535dd2ef"). InnerVolumeSpecName "kube-api-access-g2zw8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:15:10.105898 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:10.105861 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2zw8\" (UniqueName: \"kubernetes.io/projected/3d1086f5-e73f-43a9-8697-df2d535dd2ef-kube-api-access-g2zw8\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:15:10.705160 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:10.705123 2583 generic.go:356] "Generic (PLEG): container finished" podID="3d1086f5-e73f-43a9-8697-df2d535dd2ef" containerID="222e05f60868da154c6dc1ac0d418816c5ee0cb2d0f2ea33530a2b771a2ba6c4" exitCode=0 Jul 02 23:15:10.705524 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:10.705180 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-wslll" event={"ID":"3d1086f5-e73f-43a9-8697-df2d535dd2ef","Type":"ContainerDied","Data":"222e05f60868da154c6dc1ac0d418816c5ee0cb2d0f2ea33530a2b771a2ba6c4"} Jul 02 23:15:10.705524 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:10.705184 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-wslll" Jul 02 23:15:10.705524 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:10.705213 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-wslll" event={"ID":"3d1086f5-e73f-43a9-8697-df2d535dd2ef","Type":"ContainerDied","Data":"1de84adc755b9cbebebc7b1c4feab203149aebb23150ac0bd4ffc537ee32e578"} Jul 02 23:15:10.705524 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:10.705229 2583 scope.go:117] "RemoveContainer" containerID="222e05f60868da154c6dc1ac0d418816c5ee0cb2d0f2ea33530a2b771a2ba6c4" Jul 02 23:15:10.714275 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:10.714256 2583 scope.go:117] "RemoveContainer" containerID="222e05f60868da154c6dc1ac0d418816c5ee0cb2d0f2ea33530a2b771a2ba6c4" Jul 02 23:15:10.714535 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:15:10.714515 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"222e05f60868da154c6dc1ac0d418816c5ee0cb2d0f2ea33530a2b771a2ba6c4\": container with ID starting with 222e05f60868da154c6dc1ac0d418816c5ee0cb2d0f2ea33530a2b771a2ba6c4 not found: ID does not exist" containerID="222e05f60868da154c6dc1ac0d418816c5ee0cb2d0f2ea33530a2b771a2ba6c4" Jul 02 23:15:10.714611 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:10.714546 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"222e05f60868da154c6dc1ac0d418816c5ee0cb2d0f2ea33530a2b771a2ba6c4"} err="failed to get container status \"222e05f60868da154c6dc1ac0d418816c5ee0cb2d0f2ea33530a2b771a2ba6c4\": rpc error: code = NotFound desc = could not find container \"222e05f60868da154c6dc1ac0d418816c5ee0cb2d0f2ea33530a2b771a2ba6c4\": container with ID starting with 222e05f60868da154c6dc1ac0d418816c5ee0cb2d0f2ea33530a2b771a2ba6c4 not found: ID does not exist" Jul 02 23:15:10.721530 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:10.721504 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-wslll"] Jul 02 23:15:10.725162 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:10.725139 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-wslll"] Jul 02 23:15:12.533195 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:12.533160 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d1086f5-e73f-43a9-8697-df2d535dd2ef" path="/var/lib/kubelet/pods/3d1086f5-e73f-43a9-8697-df2d535dd2ef/volumes" Jul 02 23:15:33.384016 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.383980 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s8n9b"] Jul 02 23:15:33.384458 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.384306 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d1086f5-e73f-43a9-8697-df2d535dd2ef" containerName="authorino" Jul 02 23:15:33.384458 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.384317 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d1086f5-e73f-43a9-8697-df2d535dd2ef" containerName="authorino" Jul 02 23:15:33.384458 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.384387 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d1086f5-e73f-43a9-8697-df2d535dd2ef" containerName="authorino" Jul 02 23:15:33.388627 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.388603 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-s8n9b" Jul 02 23:15:33.393426 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.393397 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s8n9b"] Jul 02 23:15:33.504254 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.504221 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd8h2\" (UniqueName: \"kubernetes.io/projected/c64cfd22-5c17-4c0d-97c8-3c0324f4ce7a-kube-api-access-bd8h2\") pod \"authorino-5d4b67c446-s8n9b\" (UID: \"c64cfd22-5c17-4c0d-97c8-3c0324f4ce7a\") " pod="kuadrant-system/authorino-5d4b67c446-s8n9b" Jul 02 23:15:33.605652 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.605610 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd8h2\" (UniqueName: \"kubernetes.io/projected/c64cfd22-5c17-4c0d-97c8-3c0324f4ce7a-kube-api-access-bd8h2\") pod \"authorino-5d4b67c446-s8n9b\" (UID: \"c64cfd22-5c17-4c0d-97c8-3c0324f4ce7a\") " pod="kuadrant-system/authorino-5d4b67c446-s8n9b" Jul 02 23:15:33.611612 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.611583 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s8n9b"] Jul 02 23:15:33.611832 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:15:33.611814 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-bd8h2], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-s8n9b" podUID="c64cfd22-5c17-4c0d-97c8-3c0324f4ce7a" Jul 02 23:15:33.614909 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.614870 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd8h2\" (UniqueName: \"kubernetes.io/projected/c64cfd22-5c17-4c0d-97c8-3c0324f4ce7a-kube-api-access-bd8h2\") pod \"authorino-5d4b67c446-s8n9b\" (UID: \"c64cfd22-5c17-4c0d-97c8-3c0324f4ce7a\") " pod="kuadrant-system/authorino-5d4b67c446-s8n9b" Jul 02 23:15:33.637371 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.637285 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8dcd769df-mtsf6"] Jul 02 23:15:33.640224 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.640203 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8dcd769df-mtsf6" Jul 02 23:15:33.648168 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.648143 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8dcd769df-mtsf6"] Jul 02 23:15:33.706541 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.706499 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8h8k\" (UniqueName: \"kubernetes.io/projected/f15a01a6-d465-470f-9d66-139d56083234-kube-api-access-t8h8k\") pod \"authorino-8dcd769df-mtsf6\" (UID: \"f15a01a6-d465-470f-9d66-139d56083234\") " pod="kuadrant-system/authorino-8dcd769df-mtsf6" Jul 02 23:15:33.731348 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.731313 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8dcd769df-mtsf6"] Jul 02 23:15:33.731579 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:15:33.731561 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-t8h8k], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-8dcd769df-mtsf6" podUID="f15a01a6-d465-470f-9d66-139d56083234" Jul 02 23:15:33.758054 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.758020 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57dd9b9dfc-6qdgd"] Jul 02 23:15:33.760509 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.760489 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57dd9b9dfc-6qdgd" Jul 02 23:15:33.762680 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.762651 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 02 23:15:33.768271 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.768250 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57dd9b9dfc-6qdgd"] Jul 02 23:15:33.807485 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.807456 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8h8k\" (UniqueName: \"kubernetes.io/projected/f15a01a6-d465-470f-9d66-139d56083234-kube-api-access-t8h8k\") pod \"authorino-8dcd769df-mtsf6\" (UID: \"f15a01a6-d465-470f-9d66-139d56083234\") " pod="kuadrant-system/authorino-8dcd769df-mtsf6" Jul 02 23:15:33.810092 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.810069 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8dcd769df-mtsf6" Jul 02 23:15:33.810227 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.810069 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-s8n9b" Jul 02 23:15:33.816352 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.816327 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8dcd769df-mtsf6" Jul 02 23:15:33.816505 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.816491 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8h8k\" (UniqueName: \"kubernetes.io/projected/f15a01a6-d465-470f-9d66-139d56083234-kube-api-access-t8h8k\") pod \"authorino-8dcd769df-mtsf6\" (UID: \"f15a01a6-d465-470f-9d66-139d56083234\") " pod="kuadrant-system/authorino-8dcd769df-mtsf6" Jul 02 23:15:33.826726 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.826706 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-s8n9b" Jul 02 23:15:33.908531 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.908448 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8h8k\" (UniqueName: \"kubernetes.io/projected/f15a01a6-d465-470f-9d66-139d56083234-kube-api-access-t8h8k\") pod \"f15a01a6-d465-470f-9d66-139d56083234\" (UID: \"f15a01a6-d465-470f-9d66-139d56083234\") " Jul 02 23:15:33.908685 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.908533 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bd8h2\" (UniqueName: \"kubernetes.io/projected/c64cfd22-5c17-4c0d-97c8-3c0324f4ce7a-kube-api-access-bd8h2\") pod \"c64cfd22-5c17-4c0d-97c8-3c0324f4ce7a\" (UID: \"c64cfd22-5c17-4c0d-97c8-3c0324f4ce7a\") " Jul 02 23:15:33.908685 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.908631 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6ef1628-7f58-45d3-8138-543af0f19d8f-tls-cert\") pod \"authorino-57dd9b9dfc-6qdgd\" (UID: \"f6ef1628-7f58-45d3-8138-543af0f19d8f\") " pod="kuadrant-system/authorino-57dd9b9dfc-6qdgd" Jul 02 23:15:33.908772 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.908733 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8rg2\" (UniqueName: \"kubernetes.io/projected/f6ef1628-7f58-45d3-8138-543af0f19d8f-kube-api-access-b8rg2\") pod \"authorino-57dd9b9dfc-6qdgd\" (UID: \"f6ef1628-7f58-45d3-8138-543af0f19d8f\") " pod="kuadrant-system/authorino-57dd9b9dfc-6qdgd" Jul 02 23:15:33.910880 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.910851 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c64cfd22-5c17-4c0d-97c8-3c0324f4ce7a-kube-api-access-bd8h2" (OuterVolumeSpecName: "kube-api-access-bd8h2") pod "c64cfd22-5c17-4c0d-97c8-3c0324f4ce7a" (UID: "c64cfd22-5c17-4c0d-97c8-3c0324f4ce7a"). InnerVolumeSpecName "kube-api-access-bd8h2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:15:33.911022 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:33.910881 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f15a01a6-d465-470f-9d66-139d56083234-kube-api-access-t8h8k" (OuterVolumeSpecName: "kube-api-access-t8h8k") pod "f15a01a6-d465-470f-9d66-139d56083234" (UID: "f15a01a6-d465-470f-9d66-139d56083234"). InnerVolumeSpecName "kube-api-access-t8h8k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:15:34.009435 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:34.009397 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6ef1628-7f58-45d3-8138-543af0f19d8f-tls-cert\") pod \"authorino-57dd9b9dfc-6qdgd\" (UID: \"f6ef1628-7f58-45d3-8138-543af0f19d8f\") " pod="kuadrant-system/authorino-57dd9b9dfc-6qdgd" Jul 02 23:15:34.009600 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:34.009462 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8rg2\" (UniqueName: \"kubernetes.io/projected/f6ef1628-7f58-45d3-8138-543af0f19d8f-kube-api-access-b8rg2\") pod \"authorino-57dd9b9dfc-6qdgd\" (UID: \"f6ef1628-7f58-45d3-8138-543af0f19d8f\") " pod="kuadrant-system/authorino-57dd9b9dfc-6qdgd" Jul 02 23:15:34.009600 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:34.009516 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bd8h2\" (UniqueName: \"kubernetes.io/projected/c64cfd22-5c17-4c0d-97c8-3c0324f4ce7a-kube-api-access-bd8h2\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:15:34.009600 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:34.009526 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8h8k\" (UniqueName: \"kubernetes.io/projected/f15a01a6-d465-470f-9d66-139d56083234-kube-api-access-t8h8k\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:15:34.011984 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:34.011953 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6ef1628-7f58-45d3-8138-543af0f19d8f-tls-cert\") pod \"authorino-57dd9b9dfc-6qdgd\" (UID: \"f6ef1628-7f58-45d3-8138-543af0f19d8f\") " pod="kuadrant-system/authorino-57dd9b9dfc-6qdgd" Jul 02 23:15:34.018151 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:34.018124 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8rg2\" (UniqueName: \"kubernetes.io/projected/f6ef1628-7f58-45d3-8138-543af0f19d8f-kube-api-access-b8rg2\") pod \"authorino-57dd9b9dfc-6qdgd\" (UID: \"f6ef1628-7f58-45d3-8138-543af0f19d8f\") " pod="kuadrant-system/authorino-57dd9b9dfc-6qdgd" Jul 02 23:15:34.069944 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:34.069908 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57dd9b9dfc-6qdgd" Jul 02 23:15:34.203440 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:34.203412 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57dd9b9dfc-6qdgd"] Jul 02 23:15:34.205234 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:15:34.205208 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6ef1628_7f58_45d3_8138_543af0f19d8f.slice/crio-fee008e7ce299b518c0f789f4adc25cd25d2432fb8e9ac687ebaac93c11f6c28 WatchSource:0}: Error finding container fee008e7ce299b518c0f789f4adc25cd25d2432fb8e9ac687ebaac93c11f6c28: Status 404 returned error can't find the container with id fee008e7ce299b518c0f789f4adc25cd25d2432fb8e9ac687ebaac93c11f6c28 Jul 02 23:15:34.816362 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:34.816329 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-s8n9b" Jul 02 23:15:34.816362 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:34.816327 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57dd9b9dfc-6qdgd" event={"ID":"f6ef1628-7f58-45d3-8138-543af0f19d8f","Type":"ContainerStarted","Data":"e53de2414f2c73799e23cede6dbd1f14d91c575ab1ae605ca6dd9a4319265d00"} Jul 02 23:15:34.816808 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:34.816382 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57dd9b9dfc-6qdgd" event={"ID":"f6ef1628-7f58-45d3-8138-543af0f19d8f","Type":"ContainerStarted","Data":"fee008e7ce299b518c0f789f4adc25cd25d2432fb8e9ac687ebaac93c11f6c28"} Jul 02 23:15:34.816808 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:34.816494 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8dcd769df-mtsf6" Jul 02 23:15:34.831905 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:34.831842 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-57dd9b9dfc-6qdgd" podStartSLOduration=1.460443131 podStartE2EDuration="1.831826433s" totalImagesPullingTime="371.383302ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:15:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:15:34.829861678 +0000 UTC m=+498.852679791" watchObservedRunningTime="2026-07-02 23:15:34.831826433 +0000 UTC m=+498.854644545" Jul 02 23:15:34.854377 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:34.854315 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7td9f"] Jul 02 23:15:34.854615 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:34.854586 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-7td9f" podUID="fc58071c-cf7d-496e-a9d5-cdbee2076b87" containerName="authorino" containerID="cri-o://ffd50bf0a168c18df0f488c46fedeea203008fdc22c8db065c8da290b6682104" gracePeriod=30 Jul 02 23:15:34.867194 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:34.867159 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s8n9b"] Jul 02 23:15:34.873134 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:34.873105 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s8n9b"] Jul 02 23:15:34.890631 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:34.890591 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8dcd769df-mtsf6"] Jul 02 23:15:34.892186 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:34.892158 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8dcd769df-mtsf6"] Jul 02 23:15:35.109218 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:35.109194 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7td9f" Jul 02 23:15:35.219552 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:35.219517 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84rhf\" (UniqueName: \"kubernetes.io/projected/fc58071c-cf7d-496e-a9d5-cdbee2076b87-kube-api-access-84rhf\") pod \"fc58071c-cf7d-496e-a9d5-cdbee2076b87\" (UID: \"fc58071c-cf7d-496e-a9d5-cdbee2076b87\") " Jul 02 23:15:35.221791 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:35.221761 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc58071c-cf7d-496e-a9d5-cdbee2076b87-kube-api-access-84rhf" (OuterVolumeSpecName: "kube-api-access-84rhf") pod "fc58071c-cf7d-496e-a9d5-cdbee2076b87" (UID: "fc58071c-cf7d-496e-a9d5-cdbee2076b87"). InnerVolumeSpecName "kube-api-access-84rhf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:15:35.320399 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:35.320361 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84rhf\" (UniqueName: \"kubernetes.io/projected/fc58071c-cf7d-496e-a9d5-cdbee2076b87-kube-api-access-84rhf\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:15:35.822518 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:35.822483 2583 generic.go:356] "Generic (PLEG): container finished" podID="fc58071c-cf7d-496e-a9d5-cdbee2076b87" containerID="ffd50bf0a168c18df0f488c46fedeea203008fdc22c8db065c8da290b6682104" exitCode=0 Jul 02 23:15:35.822951 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:35.822524 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7td9f" Jul 02 23:15:35.822951 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:35.822569 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7td9f" event={"ID":"fc58071c-cf7d-496e-a9d5-cdbee2076b87","Type":"ContainerDied","Data":"ffd50bf0a168c18df0f488c46fedeea203008fdc22c8db065c8da290b6682104"} Jul 02 23:15:35.822951 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:35.822607 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7td9f" event={"ID":"fc58071c-cf7d-496e-a9d5-cdbee2076b87","Type":"ContainerDied","Data":"92f82645565a72f8bafa8dfb256e8c4da06cfc168d12b4c221a0e4563fc38119"} Jul 02 23:15:35.822951 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:35.822623 2583 scope.go:117] "RemoveContainer" containerID="ffd50bf0a168c18df0f488c46fedeea203008fdc22c8db065c8da290b6682104" Jul 02 23:15:35.831312 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:35.831289 2583 scope.go:117] "RemoveContainer" containerID="ffd50bf0a168c18df0f488c46fedeea203008fdc22c8db065c8da290b6682104" Jul 02 23:15:35.831584 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:15:35.831565 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffd50bf0a168c18df0f488c46fedeea203008fdc22c8db065c8da290b6682104\": container with ID starting with ffd50bf0a168c18df0f488c46fedeea203008fdc22c8db065c8da290b6682104 not found: ID does not exist" containerID="ffd50bf0a168c18df0f488c46fedeea203008fdc22c8db065c8da290b6682104" Jul 02 23:15:35.831627 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:35.831593 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffd50bf0a168c18df0f488c46fedeea203008fdc22c8db065c8da290b6682104"} err="failed to get container status \"ffd50bf0a168c18df0f488c46fedeea203008fdc22c8db065c8da290b6682104\": rpc error: code = NotFound desc = could not find container \"ffd50bf0a168c18df0f488c46fedeea203008fdc22c8db065c8da290b6682104\": container with ID starting with ffd50bf0a168c18df0f488c46fedeea203008fdc22c8db065c8da290b6682104 not found: ID does not exist" Jul 02 23:15:35.843113 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:35.843083 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7td9f"] Jul 02 23:15:35.845146 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:35.845122 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7td9f"] Jul 02 23:15:36.532140 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:36.532100 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c64cfd22-5c17-4c0d-97c8-3c0324f4ce7a" path="/var/lib/kubelet/pods/c64cfd22-5c17-4c0d-97c8-3c0324f4ce7a/volumes" Jul 02 23:15:36.532447 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:36.532428 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f15a01a6-d465-470f-9d66-139d56083234" path="/var/lib/kubelet/pods/f15a01a6-d465-470f-9d66-139d56083234/volumes" Jul 02 23:15:36.532678 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:15:36.532662 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc58071c-cf7d-496e-a9d5-cdbee2076b87" path="/var/lib/kubelet/pods/fc58071c-cf7d-496e-a9d5-cdbee2076b87/volumes" Jul 02 23:16:22.037277 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:22.037193 2583 generic.go:356] "Generic (PLEG): container finished" podID="35aad047-abe8-466e-9e01-ce87f1e2b080" containerID="aa4f7e01e80605439208c807f26c4e16b777a5a4db018e3907f627197a33821e" exitCode=0 Jul 02 23:16:22.037277 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:22.037267 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" event={"ID":"35aad047-abe8-466e-9e01-ce87f1e2b080","Type":"ContainerDied","Data":"aa4f7e01e80605439208c807f26c4e16b777a5a4db018e3907f627197a33821e"} Jul 02 23:16:22.037683 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:22.037631 2583 scope.go:117] "RemoveContainer" containerID="aa4f7e01e80605439208c807f26c4e16b777a5a4db018e3907f627197a33821e" Jul 02 23:16:22.038566 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:22.038542 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:16:22.038620 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:22.038608 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:16:22.038659 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:22.038642 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:16:23.043104 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:23.043072 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" event={"ID":"35aad047-abe8-466e-9e01-ce87f1e2b080","Type":"ContainerStarted","Data":"2d2da41d057d5c78b3b59039fd719774c32bb8c4b5017d480fb3b0b44eebd872"} Jul 02 23:16:23.245830 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:23.245791 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:16:23.246162 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:23.246141 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-v8h54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 02 23:16:23.246212 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:23.246197 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" podUID="35aad047-abe8-466e-9e01-ce87f1e2b080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 02 23:16:24.245944 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:24.245907 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-v8h54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 02 23:16:24.246296 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:24.245965 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" podUID="35aad047-abe8-466e-9e01-ce87f1e2b080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 02 23:16:25.246478 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:25.246429 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-v8h54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 02 23:16:25.246983 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:25.246501 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" podUID="35aad047-abe8-466e-9e01-ce87f1e2b080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 02 23:16:26.246632 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:26.246599 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-v8h54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 02 23:16:26.247040 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:26.246694 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" podUID="35aad047-abe8-466e-9e01-ce87f1e2b080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 02 23:16:27.249661 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:27.249628 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:16:27.250072 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:27.249922 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:16:27.250712 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:27.250687 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v8h54" Jul 02 23:16:47.343946 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.343903 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb"] Jul 02 23:16:47.344407 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.344295 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc58071c-cf7d-496e-a9d5-cdbee2076b87" containerName="authorino" Jul 02 23:16:47.344407 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.344308 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc58071c-cf7d-496e-a9d5-cdbee2076b87" containerName="authorino" Jul 02 23:16:47.344407 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.344357 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc58071c-cf7d-496e-a9d5-cdbee2076b87" containerName="authorino" Jul 02 23:16:47.347253 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.347232 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.350253 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.350230 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 02 23:16:47.351071 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.351043 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 02 23:16:47.351190 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.351050 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-n58qc\"" Jul 02 23:16:47.351190 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.351108 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 02 23:16:47.357175 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.357150 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb"] Jul 02 23:16:47.534395 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.534362 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/079adb89-9b45-43b7-809b-256212373341-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b88jb\" (UID: \"079adb89-9b45-43b7-809b-256212373341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.534567 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.534405 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdtff\" (UniqueName: \"kubernetes.io/projected/079adb89-9b45-43b7-809b-256212373341-kube-api-access-bdtff\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b88jb\" (UID: \"079adb89-9b45-43b7-809b-256212373341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.534567 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.534454 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/079adb89-9b45-43b7-809b-256212373341-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b88jb\" (UID: \"079adb89-9b45-43b7-809b-256212373341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.534567 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.534513 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/079adb89-9b45-43b7-809b-256212373341-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b88jb\" (UID: \"079adb89-9b45-43b7-809b-256212373341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.534567 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.534539 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/079adb89-9b45-43b7-809b-256212373341-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b88jb\" (UID: \"079adb89-9b45-43b7-809b-256212373341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.534567 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.534556 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/079adb89-9b45-43b7-809b-256212373341-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b88jb\" (UID: \"079adb89-9b45-43b7-809b-256212373341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.635571 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.635482 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/079adb89-9b45-43b7-809b-256212373341-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b88jb\" (UID: \"079adb89-9b45-43b7-809b-256212373341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.635742 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.635633 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/079adb89-9b45-43b7-809b-256212373341-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b88jb\" (UID: \"079adb89-9b45-43b7-809b-256212373341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.635742 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.635669 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/079adb89-9b45-43b7-809b-256212373341-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b88jb\" (UID: \"079adb89-9b45-43b7-809b-256212373341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.635742 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.635686 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/079adb89-9b45-43b7-809b-256212373341-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b88jb\" (UID: \"079adb89-9b45-43b7-809b-256212373341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.635742 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.635709 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/079adb89-9b45-43b7-809b-256212373341-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b88jb\" (UID: \"079adb89-9b45-43b7-809b-256212373341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.635996 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.635770 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdtff\" (UniqueName: \"kubernetes.io/projected/079adb89-9b45-43b7-809b-256212373341-kube-api-access-bdtff\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b88jb\" (UID: \"079adb89-9b45-43b7-809b-256212373341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.636212 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.636167 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/079adb89-9b45-43b7-809b-256212373341-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b88jb\" (UID: \"079adb89-9b45-43b7-809b-256212373341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.636340 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.636201 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/079adb89-9b45-43b7-809b-256212373341-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b88jb\" (UID: \"079adb89-9b45-43b7-809b-256212373341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.636340 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.636290 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/079adb89-9b45-43b7-809b-256212373341-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b88jb\" (UID: \"079adb89-9b45-43b7-809b-256212373341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.637974 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.637954 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/079adb89-9b45-43b7-809b-256212373341-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b88jb\" (UID: \"079adb89-9b45-43b7-809b-256212373341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.638752 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.638732 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/079adb89-9b45-43b7-809b-256212373341-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b88jb\" (UID: \"079adb89-9b45-43b7-809b-256212373341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.644335 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.644313 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdtff\" (UniqueName: \"kubernetes.io/projected/079adb89-9b45-43b7-809b-256212373341-kube-api-access-bdtff\") pod \"e2e-distinct-simulated-kserve-59975d69f9-b88jb\" (UID: \"079adb89-9b45-43b7-809b-256212373341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.659287 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.659262 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:47.796530 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:47.796503 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb"] Jul 02 23:16:47.798652 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:16:47.798618 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod079adb89_9b45_43b7_809b_256212373341.slice/crio-3eaef4c22694104a11972d6de065a96d0365d58d75a71586c0fcdb84b641cb20 WatchSource:0}: Error finding container 3eaef4c22694104a11972d6de065a96d0365d58d75a71586c0fcdb84b641cb20: Status 404 returned error can't find the container with id 3eaef4c22694104a11972d6de065a96d0365d58d75a71586c0fcdb84b641cb20 Jul 02 23:16:48.167488 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:48.167413 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" event={"ID":"079adb89-9b45-43b7-809b-256212373341","Type":"ContainerStarted","Data":"3eaef4c22694104a11972d6de065a96d0365d58d75a71586c0fcdb84b641cb20"} Jul 02 23:16:49.174594 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:49.174498 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" event={"ID":"079adb89-9b45-43b7-809b-256212373341","Type":"ContainerStarted","Data":"a016f5874ef842266d010b6a52415ed61fcc3ad1f2c0be524a78db1091a6998f"} Jul 02 23:16:49.194978 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:49.194919 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" podStartSLOduration=1.073137136 podStartE2EDuration="2.194877578s" totalImagesPullingTime="1.121740442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:16:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:16:49.194751571 +0000 UTC m=+573.217569685" watchObservedRunningTime="2026-07-02 23:16:49.194877578 +0000 UTC m=+573.217695690" Jul 02 23:16:57.659866 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:57.659819 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:57.660374 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:57.659919 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:57.672806 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:57.672774 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:16:58.227975 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:16:58.227940 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-b88jb" Jul 02 23:17:08.533870 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.533824 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh"] Jul 02 23:17:08.537619 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.537593 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.539861 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.539837 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 02 23:17:08.546697 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.546668 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh"] Jul 02 23:17:08.628367 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.628325 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwwjj\" (UniqueName: \"kubernetes.io/projected/1d35a04f-f08f-417d-9c8b-6507e3f2f228-kube-api-access-rwwjj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4g8gh\" (UID: \"1d35a04f-f08f-417d-9c8b-6507e3f2f228\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.628574 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.628385 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d35a04f-f08f-417d-9c8b-6507e3f2f228-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4g8gh\" (UID: \"1d35a04f-f08f-417d-9c8b-6507e3f2f228\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.628574 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.628418 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d35a04f-f08f-417d-9c8b-6507e3f2f228-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4g8gh\" (UID: \"1d35a04f-f08f-417d-9c8b-6507e3f2f228\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.628574 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.628475 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d35a04f-f08f-417d-9c8b-6507e3f2f228-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4g8gh\" (UID: \"1d35a04f-f08f-417d-9c8b-6507e3f2f228\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.628574 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.628545 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d35a04f-f08f-417d-9c8b-6507e3f2f228-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4g8gh\" (UID: \"1d35a04f-f08f-417d-9c8b-6507e3f2f228\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.628751 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.628576 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d35a04f-f08f-417d-9c8b-6507e3f2f228-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4g8gh\" (UID: \"1d35a04f-f08f-417d-9c8b-6507e3f2f228\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.729384 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.729348 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d35a04f-f08f-417d-9c8b-6507e3f2f228-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4g8gh\" (UID: \"1d35a04f-f08f-417d-9c8b-6507e3f2f228\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.729384 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.729390 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d35a04f-f08f-417d-9c8b-6507e3f2f228-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4g8gh\" (UID: \"1d35a04f-f08f-417d-9c8b-6507e3f2f228\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.729598 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.729409 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d35a04f-f08f-417d-9c8b-6507e3f2f228-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4g8gh\" (UID: \"1d35a04f-f08f-417d-9c8b-6507e3f2f228\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.729598 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.729450 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d35a04f-f08f-417d-9c8b-6507e3f2f228-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4g8gh\" (UID: \"1d35a04f-f08f-417d-9c8b-6507e3f2f228\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.729598 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.729469 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d35a04f-f08f-417d-9c8b-6507e3f2f228-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4g8gh\" (UID: \"1d35a04f-f08f-417d-9c8b-6507e3f2f228\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.729598 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.729516 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwwjj\" (UniqueName: \"kubernetes.io/projected/1d35a04f-f08f-417d-9c8b-6507e3f2f228-kube-api-access-rwwjj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4g8gh\" (UID: \"1d35a04f-f08f-417d-9c8b-6507e3f2f228\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.729833 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.729793 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d35a04f-f08f-417d-9c8b-6507e3f2f228-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4g8gh\" (UID: \"1d35a04f-f08f-417d-9c8b-6507e3f2f228\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.729931 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.729866 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d35a04f-f08f-417d-9c8b-6507e3f2f228-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4g8gh\" (UID: \"1d35a04f-f08f-417d-9c8b-6507e3f2f228\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.729998 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.729969 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d35a04f-f08f-417d-9c8b-6507e3f2f228-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4g8gh\" (UID: \"1d35a04f-f08f-417d-9c8b-6507e3f2f228\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.731848 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.731823 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d35a04f-f08f-417d-9c8b-6507e3f2f228-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4g8gh\" (UID: \"1d35a04f-f08f-417d-9c8b-6507e3f2f228\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.731982 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.731962 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d35a04f-f08f-417d-9c8b-6507e3f2f228-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4g8gh\" (UID: \"1d35a04f-f08f-417d-9c8b-6507e3f2f228\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.741908 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.741857 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwwjj\" (UniqueName: \"kubernetes.io/projected/1d35a04f-f08f-417d-9c8b-6507e3f2f228-kube-api-access-rwwjj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4g8gh\" (UID: \"1d35a04f-f08f-417d-9c8b-6507e3f2f228\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.850109 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.850060 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:08.990500 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:08.990467 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh"] Jul 02 23:17:08.992728 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:17:08.992696 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d35a04f_f08f_417d_9c8b_6507e3f2f228.slice/crio-d3169ffa0d628d347e7b24fedcd29e84739a50884c4c3e57c5fae0f3f07bcd5b WatchSource:0}: Error finding container d3169ffa0d628d347e7b24fedcd29e84739a50884c4c3e57c5fae0f3f07bcd5b: Status 404 returned error can't find the container with id d3169ffa0d628d347e7b24fedcd29e84739a50884c4c3e57c5fae0f3f07bcd5b Jul 02 23:17:09.265618 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:09.265582 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" event={"ID":"1d35a04f-f08f-417d-9c8b-6507e3f2f228","Type":"ContainerStarted","Data":"f9276da3a78c119e48e3b09055d4843947c8530aaa7a0f41ba1e996ed38cfad0"} Jul 02 23:17:09.265773 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:09.265646 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" event={"ID":"1d35a04f-f08f-417d-9c8b-6507e3f2f228","Type":"ContainerStarted","Data":"d3169ffa0d628d347e7b24fedcd29e84739a50884c4c3e57c5fae0f3f07bcd5b"} Jul 02 23:17:09.281024 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:09.280975 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" podStartSLOduration=1.104336688 podStartE2EDuration="1.280957468s" totalImagesPullingTime="176.62078ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:17:09.278778787 +0000 UTC m=+593.301596925" watchObservedRunningTime="2026-07-02 23:17:09.280957468 +0000 UTC m=+593.303775581" Jul 02 23:17:18.850862 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:18.850818 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:18.851348 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:18.850925 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:18.863230 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:18.863195 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:19.324532 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:19.324498 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4g8gh" Jul 02 23:17:22.016077 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.016039 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb"] Jul 02 23:17:22.020030 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.020005 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.022286 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.022266 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 02 23:17:22.030436 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.030407 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb"] Jul 02 23:17:22.149138 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.149104 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfea2d44-f8af-4eba-bb7f-19da7387f04f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lqrqb\" (UID: \"dfea2d44-f8af-4eba-bb7f-19da7387f04f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.149315 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.149152 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfea2d44-f8af-4eba-bb7f-19da7387f04f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lqrqb\" (UID: \"dfea2d44-f8af-4eba-bb7f-19da7387f04f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.149315 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.149284 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfea2d44-f8af-4eba-bb7f-19da7387f04f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lqrqb\" (UID: \"dfea2d44-f8af-4eba-bb7f-19da7387f04f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.149393 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.149313 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfea2d44-f8af-4eba-bb7f-19da7387f04f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lqrqb\" (UID: \"dfea2d44-f8af-4eba-bb7f-19da7387f04f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.149393 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.149334 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b889j\" (UniqueName: \"kubernetes.io/projected/dfea2d44-f8af-4eba-bb7f-19da7387f04f-kube-api-access-b889j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lqrqb\" (UID: \"dfea2d44-f8af-4eba-bb7f-19da7387f04f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.149461 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.149402 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfea2d44-f8af-4eba-bb7f-19da7387f04f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lqrqb\" (UID: \"dfea2d44-f8af-4eba-bb7f-19da7387f04f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.250692 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.250651 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfea2d44-f8af-4eba-bb7f-19da7387f04f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lqrqb\" (UID: \"dfea2d44-f8af-4eba-bb7f-19da7387f04f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.250692 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.250691 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfea2d44-f8af-4eba-bb7f-19da7387f04f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lqrqb\" (UID: \"dfea2d44-f8af-4eba-bb7f-19da7387f04f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.250988 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.250710 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b889j\" (UniqueName: \"kubernetes.io/projected/dfea2d44-f8af-4eba-bb7f-19da7387f04f-kube-api-access-b889j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lqrqb\" (UID: \"dfea2d44-f8af-4eba-bb7f-19da7387f04f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.250988 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.250730 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfea2d44-f8af-4eba-bb7f-19da7387f04f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lqrqb\" (UID: \"dfea2d44-f8af-4eba-bb7f-19da7387f04f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.250988 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.250759 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfea2d44-f8af-4eba-bb7f-19da7387f04f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lqrqb\" (UID: \"dfea2d44-f8af-4eba-bb7f-19da7387f04f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.250988 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.250798 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfea2d44-f8af-4eba-bb7f-19da7387f04f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lqrqb\" (UID: \"dfea2d44-f8af-4eba-bb7f-19da7387f04f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.251193 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.251164 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfea2d44-f8af-4eba-bb7f-19da7387f04f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lqrqb\" (UID: \"dfea2d44-f8af-4eba-bb7f-19da7387f04f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.251251 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.251174 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfea2d44-f8af-4eba-bb7f-19da7387f04f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lqrqb\" (UID: \"dfea2d44-f8af-4eba-bb7f-19da7387f04f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.251251 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.251232 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfea2d44-f8af-4eba-bb7f-19da7387f04f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lqrqb\" (UID: \"dfea2d44-f8af-4eba-bb7f-19da7387f04f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.253243 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.253218 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfea2d44-f8af-4eba-bb7f-19da7387f04f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lqrqb\" (UID: \"dfea2d44-f8af-4eba-bb7f-19da7387f04f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.253494 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.253474 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfea2d44-f8af-4eba-bb7f-19da7387f04f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lqrqb\" (UID: \"dfea2d44-f8af-4eba-bb7f-19da7387f04f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.259442 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.259416 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b889j\" (UniqueName: \"kubernetes.io/projected/dfea2d44-f8af-4eba-bb7f-19da7387f04f-kube-api-access-b889j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lqrqb\" (UID: \"dfea2d44-f8af-4eba-bb7f-19da7387f04f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.330968 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.330881 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:22.478623 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.478588 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb"] Jul 02 23:17:22.480928 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:17:22.480455 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfea2d44_f8af_4eba_bb7f_19da7387f04f.slice/crio-21cd0c854ee0af75115e1c6c00520c26cd47bd4532a98af35f2c43f7e4276e92 WatchSource:0}: Error finding container 21cd0c854ee0af75115e1c6c00520c26cd47bd4532a98af35f2c43f7e4276e92: Status 404 returned error can't find the container with id 21cd0c854ee0af75115e1c6c00520c26cd47bd4532a98af35f2c43f7e4276e92 Jul 02 23:17:22.482296 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:22.482276 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:17:23.334618 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:23.334579 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" event={"ID":"dfea2d44-f8af-4eba-bb7f-19da7387f04f","Type":"ContainerStarted","Data":"5dc3dc4b0f4be37d6707651d399fbd9ed4c28a6991d21295efae5b13a3fefe01"} Jul 02 23:17:23.335128 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:23.334630 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" event={"ID":"dfea2d44-f8af-4eba-bb7f-19da7387f04f","Type":"ContainerStarted","Data":"21cd0c854ee0af75115e1c6c00520c26cd47bd4532a98af35f2c43f7e4276e92"} Jul 02 23:17:23.382170 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:23.382110 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" podStartSLOduration=2.212990893 podStartE2EDuration="2.382091375s" totalImagesPullingTime="169.100482ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:17:23.379557545 +0000 UTC m=+607.402375657" watchObservedRunningTime="2026-07-02 23:17:23.382091375 +0000 UTC m=+607.404909488" Jul 02 23:17:32.331653 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:32.331608 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:32.332215 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:32.331750 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:32.344072 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:32.344042 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:32.392989 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:32.392964 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lqrqb" Jul 02 23:17:39.228519 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.228435 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8"] Jul 02 23:17:39.231684 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.231624 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.234009 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.233986 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 02 23:17:39.240983 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.240951 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8"] Jul 02 23:17:39.303233 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.303184 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/921fd0f0-6306-4f0d-b8d3-4279c8014f1f-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8\" (UID: \"921fd0f0-6306-4f0d-b8d3-4279c8014f1f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.303428 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.303258 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bf4d\" (UniqueName: \"kubernetes.io/projected/921fd0f0-6306-4f0d-b8d3-4279c8014f1f-kube-api-access-6bf4d\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8\" (UID: \"921fd0f0-6306-4f0d-b8d3-4279c8014f1f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.303428 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.303293 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/921fd0f0-6306-4f0d-b8d3-4279c8014f1f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8\" (UID: \"921fd0f0-6306-4f0d-b8d3-4279c8014f1f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.303428 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.303310 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/921fd0f0-6306-4f0d-b8d3-4279c8014f1f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8\" (UID: \"921fd0f0-6306-4f0d-b8d3-4279c8014f1f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.303428 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.303410 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/921fd0f0-6306-4f0d-b8d3-4279c8014f1f-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8\" (UID: \"921fd0f0-6306-4f0d-b8d3-4279c8014f1f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.303608 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.303448 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/921fd0f0-6306-4f0d-b8d3-4279c8014f1f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8\" (UID: \"921fd0f0-6306-4f0d-b8d3-4279c8014f1f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.404203 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.404144 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/921fd0f0-6306-4f0d-b8d3-4279c8014f1f-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8\" (UID: \"921fd0f0-6306-4f0d-b8d3-4279c8014f1f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.404385 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.404209 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/921fd0f0-6306-4f0d-b8d3-4279c8014f1f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8\" (UID: \"921fd0f0-6306-4f0d-b8d3-4279c8014f1f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.404385 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.404260 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/921fd0f0-6306-4f0d-b8d3-4279c8014f1f-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8\" (UID: \"921fd0f0-6306-4f0d-b8d3-4279c8014f1f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.404385 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.404343 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bf4d\" (UniqueName: \"kubernetes.io/projected/921fd0f0-6306-4f0d-b8d3-4279c8014f1f-kube-api-access-6bf4d\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8\" (UID: \"921fd0f0-6306-4f0d-b8d3-4279c8014f1f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.404538 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.404402 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/921fd0f0-6306-4f0d-b8d3-4279c8014f1f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8\" (UID: \"921fd0f0-6306-4f0d-b8d3-4279c8014f1f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.404538 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.404424 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/921fd0f0-6306-4f0d-b8d3-4279c8014f1f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8\" (UID: \"921fd0f0-6306-4f0d-b8d3-4279c8014f1f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.404634 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.404599 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/921fd0f0-6306-4f0d-b8d3-4279c8014f1f-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8\" (UID: \"921fd0f0-6306-4f0d-b8d3-4279c8014f1f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.404687 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.404659 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/921fd0f0-6306-4f0d-b8d3-4279c8014f1f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8\" (UID: \"921fd0f0-6306-4f0d-b8d3-4279c8014f1f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.404910 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.404874 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/921fd0f0-6306-4f0d-b8d3-4279c8014f1f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8\" (UID: \"921fd0f0-6306-4f0d-b8d3-4279c8014f1f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.406971 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.406947 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/921fd0f0-6306-4f0d-b8d3-4279c8014f1f-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8\" (UID: \"921fd0f0-6306-4f0d-b8d3-4279c8014f1f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.407240 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.407221 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/921fd0f0-6306-4f0d-b8d3-4279c8014f1f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8\" (UID: \"921fd0f0-6306-4f0d-b8d3-4279c8014f1f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.412052 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.412028 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bf4d\" (UniqueName: \"kubernetes.io/projected/921fd0f0-6306-4f0d-b8d3-4279c8014f1f-kube-api-access-6bf4d\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8\" (UID: \"921fd0f0-6306-4f0d-b8d3-4279c8014f1f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.542420 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.542330 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:39.680956 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:39.680925 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8"] Jul 02 23:17:39.682823 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:17:39.682796 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod921fd0f0_6306_4f0d_b8d3_4279c8014f1f.slice/crio-0b4b9495026b6a59559328165c3e48178b5524cbec551aeeac017cf40af55db4 WatchSource:0}: Error finding container 0b4b9495026b6a59559328165c3e48178b5524cbec551aeeac017cf40af55db4: Status 404 returned error can't find the container with id 0b4b9495026b6a59559328165c3e48178b5524cbec551aeeac017cf40af55db4 Jul 02 23:17:40.421188 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:40.421151 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" event={"ID":"921fd0f0-6306-4f0d-b8d3-4279c8014f1f","Type":"ContainerStarted","Data":"a90a3f2b3192b5809da74efdd42d5fb0586001d7dea7827350b6695851429d21"} Jul 02 23:17:40.421188 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:40.421193 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" event={"ID":"921fd0f0-6306-4f0d-b8d3-4279c8014f1f","Type":"ContainerStarted","Data":"0b4b9495026b6a59559328165c3e48178b5524cbec551aeeac017cf40af55db4"} Jul 02 23:17:40.439004 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:40.438949 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" podStartSLOduration=1.268741737 podStartE2EDuration="1.438933877s" totalImagesPullingTime="170.19214ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:17:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:17:40.437847069 +0000 UTC m=+624.460665183" watchObservedRunningTime="2026-07-02 23:17:40.438933877 +0000 UTC m=+624.461751989" Jul 02 23:17:49.543525 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:49.543481 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:49.543935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:49.543542 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:49.556217 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:49.556189 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:17:50.479624 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:17:50.479592 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8" Jul 02 23:18:50.220042 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:50.220002 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5b754c5d8d-vql6w"] Jul 02 23:18:50.223524 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:50.223497 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b754c5d8d-vql6w" Jul 02 23:18:50.231605 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:50.231572 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5b754c5d8d-vql6w"] Jul 02 23:18:50.341300 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:50.341268 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbdb9\" (UniqueName: \"kubernetes.io/projected/d2258cf2-7688-4cd2-ade2-01163ffd216a-kube-api-access-rbdb9\") pod \"authorino-5b754c5d8d-vql6w\" (UID: \"d2258cf2-7688-4cd2-ade2-01163ffd216a\") " pod="kuadrant-system/authorino-5b754c5d8d-vql6w" Jul 02 23:18:50.341460 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:50.341309 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d2258cf2-7688-4cd2-ade2-01163ffd216a-tls-cert\") pod \"authorino-5b754c5d8d-vql6w\" (UID: \"d2258cf2-7688-4cd2-ade2-01163ffd216a\") " pod="kuadrant-system/authorino-5b754c5d8d-vql6w" Jul 02 23:18:50.442202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:50.442155 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbdb9\" (UniqueName: \"kubernetes.io/projected/d2258cf2-7688-4cd2-ade2-01163ffd216a-kube-api-access-rbdb9\") pod \"authorino-5b754c5d8d-vql6w\" (UID: \"d2258cf2-7688-4cd2-ade2-01163ffd216a\") " pod="kuadrant-system/authorino-5b754c5d8d-vql6w" Jul 02 23:18:50.442202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:50.442208 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d2258cf2-7688-4cd2-ade2-01163ffd216a-tls-cert\") pod \"authorino-5b754c5d8d-vql6w\" (UID: \"d2258cf2-7688-4cd2-ade2-01163ffd216a\") " pod="kuadrant-system/authorino-5b754c5d8d-vql6w" Jul 02 23:18:50.444704 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:50.444682 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d2258cf2-7688-4cd2-ade2-01163ffd216a-tls-cert\") pod \"authorino-5b754c5d8d-vql6w\" (UID: \"d2258cf2-7688-4cd2-ade2-01163ffd216a\") " pod="kuadrant-system/authorino-5b754c5d8d-vql6w" Jul 02 23:18:50.450098 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:50.450071 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbdb9\" (UniqueName: \"kubernetes.io/projected/d2258cf2-7688-4cd2-ade2-01163ffd216a-kube-api-access-rbdb9\") pod \"authorino-5b754c5d8d-vql6w\" (UID: \"d2258cf2-7688-4cd2-ade2-01163ffd216a\") " pod="kuadrant-system/authorino-5b754c5d8d-vql6w" Jul 02 23:18:50.534738 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:50.534669 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b754c5d8d-vql6w" Jul 02 23:18:50.675171 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:50.675137 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5b754c5d8d-vql6w"] Jul 02 23:18:50.678449 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:18:50.678408 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2258cf2_7688_4cd2_ade2_01163ffd216a.slice/crio-e3b1bb19c17d5719229a936327c86c1d7d6b78e5a351930ca8b2d39b960cfea3 WatchSource:0}: Error finding container e3b1bb19c17d5719229a936327c86c1d7d6b78e5a351930ca8b2d39b960cfea3: Status 404 returned error can't find the container with id e3b1bb19c17d5719229a936327c86c1d7d6b78e5a351930ca8b2d39b960cfea3 Jul 02 23:18:50.751990 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:50.751940 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b754c5d8d-vql6w" event={"ID":"d2258cf2-7688-4cd2-ade2-01163ffd216a","Type":"ContainerStarted","Data":"e3b1bb19c17d5719229a936327c86c1d7d6b78e5a351930ca8b2d39b960cfea3"} Jul 02 23:18:51.759882 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:51.759780 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b754c5d8d-vql6w" event={"ID":"d2258cf2-7688-4cd2-ade2-01163ffd216a","Type":"ContainerStarted","Data":"609c50665de88981e1b9191921e06b7edb072d9799baa3d43d2bf1b93b87d278"} Jul 02 23:18:51.780064 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:51.779995 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5b754c5d8d-vql6w" podStartSLOduration=1.0729621 podStartE2EDuration="1.779956992s" totalImagesPullingTime="706.994892ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:18:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:18:51.777312473 +0000 UTC m=+695.800130588" watchObservedRunningTime="2026-07-02 23:18:51.779956992 +0000 UTC m=+695.802775107" Jul 02 23:18:51.808307 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:51.808266 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57dd9b9dfc-6qdgd"] Jul 02 23:18:51.808520 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:51.808476 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-57dd9b9dfc-6qdgd" podUID="f6ef1628-7f58-45d3-8138-543af0f19d8f" containerName="authorino" containerID="cri-o://e53de2414f2c73799e23cede6dbd1f14d91c575ab1ae605ca6dd9a4319265d00" gracePeriod=30 Jul 02 23:18:52.077160 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:52.077127 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57dd9b9dfc-6qdgd" Jul 02 23:18:52.157332 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:52.157294 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6ef1628-7f58-45d3-8138-543af0f19d8f-tls-cert\") pod \"f6ef1628-7f58-45d3-8138-543af0f19d8f\" (UID: \"f6ef1628-7f58-45d3-8138-543af0f19d8f\") " Jul 02 23:18:52.157517 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:52.157351 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8rg2\" (UniqueName: \"kubernetes.io/projected/f6ef1628-7f58-45d3-8138-543af0f19d8f-kube-api-access-b8rg2\") pod \"f6ef1628-7f58-45d3-8138-543af0f19d8f\" (UID: \"f6ef1628-7f58-45d3-8138-543af0f19d8f\") " Jul 02 23:18:52.159566 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:52.159524 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6ef1628-7f58-45d3-8138-543af0f19d8f-kube-api-access-b8rg2" (OuterVolumeSpecName: "kube-api-access-b8rg2") pod "f6ef1628-7f58-45d3-8138-543af0f19d8f" (UID: "f6ef1628-7f58-45d3-8138-543af0f19d8f"). InnerVolumeSpecName "kube-api-access-b8rg2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:18:52.168640 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:52.168597 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6ef1628-7f58-45d3-8138-543af0f19d8f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f6ef1628-7f58-45d3-8138-543af0f19d8f" (UID: "f6ef1628-7f58-45d3-8138-543af0f19d8f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:18:52.258442 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:52.258410 2583 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6ef1628-7f58-45d3-8138-543af0f19d8f-tls-cert\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:18:52.258442 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:52.258439 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8rg2\" (UniqueName: \"kubernetes.io/projected/f6ef1628-7f58-45d3-8138-543af0f19d8f-kube-api-access-b8rg2\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:18:52.766382 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:52.766283 2583 generic.go:356] "Generic (PLEG): container finished" podID="f6ef1628-7f58-45d3-8138-543af0f19d8f" containerID="e53de2414f2c73799e23cede6dbd1f14d91c575ab1ae605ca6dd9a4319265d00" exitCode=0 Jul 02 23:18:52.766382 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:52.766333 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57dd9b9dfc-6qdgd" Jul 02 23:18:52.766952 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:52.766327 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57dd9b9dfc-6qdgd" event={"ID":"f6ef1628-7f58-45d3-8138-543af0f19d8f","Type":"ContainerDied","Data":"e53de2414f2c73799e23cede6dbd1f14d91c575ab1ae605ca6dd9a4319265d00"} Jul 02 23:18:52.766952 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:52.766440 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57dd9b9dfc-6qdgd" event={"ID":"f6ef1628-7f58-45d3-8138-543af0f19d8f","Type":"ContainerDied","Data":"fee008e7ce299b518c0f789f4adc25cd25d2432fb8e9ac687ebaac93c11f6c28"} Jul 02 23:18:52.766952 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:52.766469 2583 scope.go:117] "RemoveContainer" containerID="e53de2414f2c73799e23cede6dbd1f14d91c575ab1ae605ca6dd9a4319265d00" Jul 02 23:18:52.776007 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:52.775978 2583 scope.go:117] "RemoveContainer" containerID="e53de2414f2c73799e23cede6dbd1f14d91c575ab1ae605ca6dd9a4319265d00" Jul 02 23:18:52.776382 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:18:52.776363 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e53de2414f2c73799e23cede6dbd1f14d91c575ab1ae605ca6dd9a4319265d00\": container with ID starting with e53de2414f2c73799e23cede6dbd1f14d91c575ab1ae605ca6dd9a4319265d00 not found: ID does not exist" containerID="e53de2414f2c73799e23cede6dbd1f14d91c575ab1ae605ca6dd9a4319265d00" Jul 02 23:18:52.776468 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:52.776392 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e53de2414f2c73799e23cede6dbd1f14d91c575ab1ae605ca6dd9a4319265d00"} err="failed to get container status \"e53de2414f2c73799e23cede6dbd1f14d91c575ab1ae605ca6dd9a4319265d00\": rpc error: code = NotFound desc = could not find container \"e53de2414f2c73799e23cede6dbd1f14d91c575ab1ae605ca6dd9a4319265d00\": container with ID starting with e53de2414f2c73799e23cede6dbd1f14d91c575ab1ae605ca6dd9a4319265d00 not found: ID does not exist" Jul 02 23:18:52.784167 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:52.784139 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57dd9b9dfc-6qdgd"] Jul 02 23:18:52.786843 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:52.786814 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57dd9b9dfc-6qdgd"] Jul 02 23:18:54.532802 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:18:54.532768 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6ef1628-7f58-45d3-8138-543af0f19d8f" path="/var/lib/kubelet/pods/f6ef1628-7f58-45d3-8138-543af0f19d8f/volumes" Jul 02 23:20:16.498578 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:20:16.498542 2583 scope.go:117] "RemoveContainer" containerID="4d5f0f27393dfa4fd0c95d2a2ef0c3f5bd344d07b22c36fe3405e2f2adc0d639" Jul 02 23:24:43.703320 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:24:43.703267 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg"] Jul 02 23:24:43.703827 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:24:43.703559 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg" podUID="e1642df8-ac92-496a-b122-429dfabdba60" containerName="manager" containerID="cri-o://2f7e8cb7c6ae67922b1f6c8297cb98a4b75efa63dcdc58b1a72d314ba6bc2fb3" gracePeriod=10 Jul 02 23:24:46.869449 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:24:46.869418 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg" Jul 02 23:24:46.974663 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:24:46.974568 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1642df8-ac92-496a-b122-429dfabdba60-extensions-socket-volume\") pod \"e1642df8-ac92-496a-b122-429dfabdba60\" (UID: \"e1642df8-ac92-496a-b122-429dfabdba60\") " Jul 02 23:24:46.974839 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:24:46.974692 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkzrb\" (UniqueName: \"kubernetes.io/projected/e1642df8-ac92-496a-b122-429dfabdba60-kube-api-access-jkzrb\") pod \"e1642df8-ac92-496a-b122-429dfabdba60\" (UID: \"e1642df8-ac92-496a-b122-429dfabdba60\") " Jul 02 23:24:46.975044 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:24:46.975019 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1642df8-ac92-496a-b122-429dfabdba60-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e1642df8-ac92-496a-b122-429dfabdba60" (UID: "e1642df8-ac92-496a-b122-429dfabdba60"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:24:46.976882 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:24:46.976852 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1642df8-ac92-496a-b122-429dfabdba60-kube-api-access-jkzrb" (OuterVolumeSpecName: "kube-api-access-jkzrb") pod "e1642df8-ac92-496a-b122-429dfabdba60" (UID: "e1642df8-ac92-496a-b122-429dfabdba60"). InnerVolumeSpecName "kube-api-access-jkzrb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:24:47.075346 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:24:47.075305 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkzrb\" (UniqueName: \"kubernetes.io/projected/e1642df8-ac92-496a-b122-429dfabdba60-kube-api-access-jkzrb\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:24:47.075346 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:24:47.075337 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1642df8-ac92-496a-b122-429dfabdba60-extensions-socket-volume\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:24:47.469555 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:24:47.469516 2583 generic.go:356] "Generic (PLEG): container finished" podID="e1642df8-ac92-496a-b122-429dfabdba60" containerID="2f7e8cb7c6ae67922b1f6c8297cb98a4b75efa63dcdc58b1a72d314ba6bc2fb3" exitCode=0 Jul 02 23:24:47.469555 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:24:47.469559 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg" Jul 02 23:24:47.469790 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:24:47.469594 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg" event={"ID":"e1642df8-ac92-496a-b122-429dfabdba60","Type":"ContainerDied","Data":"2f7e8cb7c6ae67922b1f6c8297cb98a4b75efa63dcdc58b1a72d314ba6bc2fb3"} Jul 02 23:24:47.469790 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:24:47.469648 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg" event={"ID":"e1642df8-ac92-496a-b122-429dfabdba60","Type":"ContainerDied","Data":"dad28abbd24067c0f23ced0f519a0d70a9f9333c0abeaa0d5596fb74cad14e03"} Jul 02 23:24:47.469790 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:24:47.469672 2583 scope.go:117] "RemoveContainer" containerID="2f7e8cb7c6ae67922b1f6c8297cb98a4b75efa63dcdc58b1a72d314ba6bc2fb3" Jul 02 23:24:47.479090 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:24:47.479072 2583 scope.go:117] "RemoveContainer" containerID="2f7e8cb7c6ae67922b1f6c8297cb98a4b75efa63dcdc58b1a72d314ba6bc2fb3" Jul 02 23:24:47.479381 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:24:47.479361 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f7e8cb7c6ae67922b1f6c8297cb98a4b75efa63dcdc58b1a72d314ba6bc2fb3\": container with ID starting with 2f7e8cb7c6ae67922b1f6c8297cb98a4b75efa63dcdc58b1a72d314ba6bc2fb3 not found: ID does not exist" containerID="2f7e8cb7c6ae67922b1f6c8297cb98a4b75efa63dcdc58b1a72d314ba6bc2fb3" Jul 02 23:24:47.479447 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:24:47.479394 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f7e8cb7c6ae67922b1f6c8297cb98a4b75efa63dcdc58b1a72d314ba6bc2fb3"} err="failed to get container status \"2f7e8cb7c6ae67922b1f6c8297cb98a4b75efa63dcdc58b1a72d314ba6bc2fb3\": rpc error: code = NotFound desc = could not find container \"2f7e8cb7c6ae67922b1f6c8297cb98a4b75efa63dcdc58b1a72d314ba6bc2fb3\": container with ID starting with 2f7e8cb7c6ae67922b1f6c8297cb98a4b75efa63dcdc58b1a72d314ba6bc2fb3 not found: ID does not exist" Jul 02 23:24:47.489925 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:24:47.489884 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg"] Jul 02 23:24:47.497610 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:24:47.497586 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7jjsg"] Jul 02 23:24:48.532246 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:24:48.532212 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1642df8-ac92-496a-b122-429dfabdba60" path="/var/lib/kubelet/pods/e1642df8-ac92-496a-b122-429dfabdba60/volumes" Jul 02 23:30:00.137732 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:00.137637 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29717250-kckqj"] Jul 02 23:30:00.140355 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:00.138164 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6ef1628-7f58-45d3-8138-543af0f19d8f" containerName="authorino" Jul 02 23:30:00.140355 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:00.138181 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6ef1628-7f58-45d3-8138-543af0f19d8f" containerName="authorino" Jul 02 23:30:00.140355 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:00.138196 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1642df8-ac92-496a-b122-429dfabdba60" containerName="manager" Jul 02 23:30:00.140355 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:00.138202 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1642df8-ac92-496a-b122-429dfabdba60" containerName="manager" Jul 02 23:30:00.140355 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:00.138265 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6ef1628-7f58-45d3-8138-543af0f19d8f" containerName="authorino" Jul 02 23:30:00.140355 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:00.138274 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1642df8-ac92-496a-b122-429dfabdba60" containerName="manager" Jul 02 23:30:00.141190 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:00.141169 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717250-kckqj" Jul 02 23:30:00.143413 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:00.143394 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-c9hnp\"" Jul 02 23:30:00.154065 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:00.154041 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717250-kckqj"] Jul 02 23:30:00.320181 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:00.320136 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbmtb\" (UniqueName: \"kubernetes.io/projected/24e21038-26ea-4a86-a69c-45b52904e31b-kube-api-access-sbmtb\") pod \"maas-api-key-cleanup-29717250-kckqj\" (UID: \"24e21038-26ea-4a86-a69c-45b52904e31b\") " pod="opendatahub/maas-api-key-cleanup-29717250-kckqj" Jul 02 23:30:00.421677 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:00.421575 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbmtb\" (UniqueName: \"kubernetes.io/projected/24e21038-26ea-4a86-a69c-45b52904e31b-kube-api-access-sbmtb\") pod \"maas-api-key-cleanup-29717250-kckqj\" (UID: \"24e21038-26ea-4a86-a69c-45b52904e31b\") " pod="opendatahub/maas-api-key-cleanup-29717250-kckqj" Jul 02 23:30:00.429710 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:00.429679 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbmtb\" (UniqueName: \"kubernetes.io/projected/24e21038-26ea-4a86-a69c-45b52904e31b-kube-api-access-sbmtb\") pod \"maas-api-key-cleanup-29717250-kckqj\" (UID: \"24e21038-26ea-4a86-a69c-45b52904e31b\") " pod="opendatahub/maas-api-key-cleanup-29717250-kckqj" Jul 02 23:30:00.451503 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:00.451466 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717250-kckqj" Jul 02 23:30:00.589176 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:00.589150 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717250-kckqj"] Jul 02 23:30:00.591824 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:30:00.591799 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24e21038_26ea_4a86_a69c_45b52904e31b.slice/crio-b7644555b992ccb3de7ad7b06d754706930be933564de18119ae9541c2caa61c WatchSource:0}: Error finding container b7644555b992ccb3de7ad7b06d754706930be933564de18119ae9541c2caa61c: Status 404 returned error can't find the container with id b7644555b992ccb3de7ad7b06d754706930be933564de18119ae9541c2caa61c Jul 02 23:30:00.594127 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:00.594109 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:30:00.914123 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:00.914086 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717250-kckqj" event={"ID":"24e21038-26ea-4a86-a69c-45b52904e31b","Type":"ContainerStarted","Data":"b7644555b992ccb3de7ad7b06d754706930be933564de18119ae9541c2caa61c"} Jul 02 23:30:03.932731 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:03.932690 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717250-kckqj" event={"ID":"24e21038-26ea-4a86-a69c-45b52904e31b","Type":"ContainerStarted","Data":"41dc6bbf2dd4843fd41068d92f2ecd12f58c0569ec13c60317963e821e2d4542"} Jul 02 23:30:03.948337 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:03.948293 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29717250-kckqj" podStartSLOduration=1.623788357 podStartE2EDuration="3.948276426s" totalImagesPullingTime="2.324488069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:30:03.945867342 +0000 UTC m=+1367.968685456" watchObservedRunningTime="2026-07-02 23:30:03.948276426 +0000 UTC m=+1367.971094539" Jul 02 23:30:04.938172 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:04.938137 2583 generic.go:356] "Generic (PLEG): container finished" podID="24e21038-26ea-4a86-a69c-45b52904e31b" containerID="41dc6bbf2dd4843fd41068d92f2ecd12f58c0569ec13c60317963e821e2d4542" exitCode=0 Jul 02 23:30:04.938561 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:04.938188 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717250-kckqj" event={"ID":"24e21038-26ea-4a86-a69c-45b52904e31b","Type":"ContainerDied","Data":"41dc6bbf2dd4843fd41068d92f2ecd12f58c0569ec13c60317963e821e2d4542"} Jul 02 23:30:06.079234 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:06.079203 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717250-kckqj" Jul 02 23:30:06.169797 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:06.169760 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbmtb\" (UniqueName: \"kubernetes.io/projected/24e21038-26ea-4a86-a69c-45b52904e31b-kube-api-access-sbmtb\") pod \"24e21038-26ea-4a86-a69c-45b52904e31b\" (UID: \"24e21038-26ea-4a86-a69c-45b52904e31b\") " Jul 02 23:30:06.172108 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:06.172070 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24e21038-26ea-4a86-a69c-45b52904e31b-kube-api-access-sbmtb" (OuterVolumeSpecName: "kube-api-access-sbmtb") pod "24e21038-26ea-4a86-a69c-45b52904e31b" (UID: "24e21038-26ea-4a86-a69c-45b52904e31b"). InnerVolumeSpecName "kube-api-access-sbmtb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:30:06.270844 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:06.270756 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbmtb\" (UniqueName: \"kubernetes.io/projected/24e21038-26ea-4a86-a69c-45b52904e31b-kube-api-access-sbmtb\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:30:06.951731 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:06.951698 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717250-kckqj" Jul 02 23:30:06.951935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:06.951697 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717250-kckqj" event={"ID":"24e21038-26ea-4a86-a69c-45b52904e31b","Type":"ContainerDied","Data":"b7644555b992ccb3de7ad7b06d754706930be933564de18119ae9541c2caa61c"} Jul 02 23:30:06.951935 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:30:06.951852 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7644555b992ccb3de7ad7b06d754706930be933564de18119ae9541c2caa61c" Jul 02 23:31:57.118333 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.118299 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9"] Jul 02 23:31:57.118812 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.118663 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24e21038-26ea-4a86-a69c-45b52904e31b" containerName="cleanup" Jul 02 23:31:57.118812 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.118674 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="24e21038-26ea-4a86-a69c-45b52904e31b" containerName="cleanup" Jul 02 23:31:57.118812 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.118727 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="24e21038-26ea-4a86-a69c-45b52904e31b" containerName="cleanup" Jul 02 23:31:57.121592 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.121573 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.123767 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.123747 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4f93315a-openshift-default-dockercfg-l6222\"" Jul 02 23:31:57.143504 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.143472 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9"] Jul 02 23:31:57.167612 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.167573 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-workload-socket\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.167612 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.167615 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-envoy\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.167803 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.167635 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-data\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.167803 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.167700 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-podinfo\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.167803 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.167731 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istiod-ca-cert\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.167803 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.167765 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-credential-socket\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.167803 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.167797 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-token\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.167996 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.167841 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxlb7\" (UniqueName: \"kubernetes.io/projected/933c9d0a-193f-4e39-beb5-128f2eca3a6c-kube-api-access-sxlb7\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.167996 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.167867 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-workload-certs\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.268426 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.268390 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-workload-socket\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.268595 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.268437 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-envoy\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.268595 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.268469 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-data\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.268595 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.268507 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-podinfo\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.268595 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.268531 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istiod-ca-cert\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.268595 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.268574 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-credential-socket\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.268850 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.268720 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-token\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.268850 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.268773 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxlb7\" (UniqueName: \"kubernetes.io/projected/933c9d0a-193f-4e39-beb5-128f2eca3a6c-kube-api-access-sxlb7\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.268850 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.268812 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-workload-certs\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.268850 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.268844 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-workload-socket\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.269106 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.268972 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-data\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.269173 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.269127 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-credential-socket\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.269351 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.269321 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-workload-certs\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.269535 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.269512 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istiod-ca-cert\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.271052 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.271031 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-envoy\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.271265 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.271242 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-podinfo\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.276061 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.276033 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-token\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.276152 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.276130 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxlb7\" (UniqueName: \"kubernetes.io/projected/933c9d0a-193f-4e39-beb5-128f2eca3a6c-kube-api-access-sxlb7\") pod \"e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.432274 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.432187 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:57.573234 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.573208 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9"] Jul 02 23:31:57.575681 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:31:57.575638 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod933c9d0a_193f_4e39_beb5_128f2eca3a6c.slice/crio-6a876493cd8c1656780bf36129c1e61f4bfc6b98123fcbb815eb591cf1d98035 WatchSource:0}: Error finding container 6a876493cd8c1656780bf36129c1e61f4bfc6b98123fcbb815eb591cf1d98035: Status 404 returned error can't find the container with id 6a876493cd8c1656780bf36129c1e61f4bfc6b98123fcbb815eb591cf1d98035 Jul 02 23:31:57.577777 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.577740 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:31:57.577865 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.577811 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:31:57.577865 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:57.577840 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:31:58.477301 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:58.477253 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" event={"ID":"933c9d0a-193f-4e39-beb5-128f2eca3a6c","Type":"ContainerStarted","Data":"353ed944b103a036830516b821cb6bc723da601906457da3c82d1ba778055b08"} Jul 02 23:31:58.477711 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:58.477365 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" event={"ID":"933c9d0a-193f-4e39-beb5-128f2eca3a6c","Type":"ContainerStarted","Data":"6a876493cd8c1656780bf36129c1e61f4bfc6b98123fcbb815eb591cf1d98035"} Jul 02 23:31:58.526907 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:58.526820 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" podStartSLOduration=1.526798906 podStartE2EDuration="1.526798906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:31:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:31:58.525523956 +0000 UTC m=+1482.548342080" watchObservedRunningTime="2026-07-02 23:31:58.526798906 +0000 UTC m=+1482.549617021" Jul 02 23:31:59.433091 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:59.433050 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:59.437938 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:59.437910 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:59.482512 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:59.482481 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:31:59.483630 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:31:59.483608 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:32:03.612381 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:03.612339 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9"] Jul 02 23:32:03.612805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:03.612550 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" podUID="933c9d0a-193f-4e39-beb5-128f2eca3a6c" containerName="istio-proxy" containerID="cri-o://353ed944b103a036830516b821cb6bc723da601906457da3c82d1ba778055b08" gracePeriod=30 Jul 02 23:32:09.718283 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.718249 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb"] Jul 02 23:32:09.722012 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.721983 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.724303 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.724275 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-7dd7cae8-openshift-default-dockercfg-nwk5t\"" Jul 02 23:32:09.733640 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.733616 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb"] Jul 02 23:32:09.790576 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.790537 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-envoy\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.790718 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.790699 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wncc\" (UniqueName: \"kubernetes.io/projected/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-kube-api-access-9wncc\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.790772 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.790739 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-data\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.790772 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.790759 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-podinfo\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.790904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.790786 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-token\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.790904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.790824 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-credential-socket\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.790904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.790871 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-workload-socket\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.791045 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.790914 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-workload-certs\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.791045 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.790938 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istiod-ca-cert\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.866959 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.866929 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:32:09.892450 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.892357 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-data\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.892450 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.892391 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-podinfo\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.892450 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.892415 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-token\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.892450 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.892450 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-credential-socket\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.892775 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.892468 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-workload-socket\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.892775 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.892486 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-workload-certs\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.892775 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.892508 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istiod-ca-cert\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.892775 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.892540 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-envoy\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.892775 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.892626 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wncc\" (UniqueName: \"kubernetes.io/projected/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-kube-api-access-9wncc\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.893052 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.892837 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-data\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.893052 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.892988 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-credential-socket\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.893052 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.893038 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-workload-certs\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.893202 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.893106 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-workload-socket\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.893350 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.893330 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istiod-ca-cert\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.895042 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.895026 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-envoy\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.895144 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.895125 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-podinfo\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.915207 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.915170 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-token\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.916480 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.916452 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wncc\" (UniqueName: \"kubernetes.io/projected/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-kube-api-access-9wncc\") pod \"e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:09.993284 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.993247 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-podinfo\") pod \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " Jul 02 23:32:09.993457 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.993341 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-credential-socket\") pod \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " Jul 02 23:32:09.993457 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.993359 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istiod-ca-cert\") pod \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " Jul 02 23:32:09.993457 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.993375 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-token\") pod \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " Jul 02 23:32:09.993457 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.993394 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxlb7\" (UniqueName: \"kubernetes.io/projected/933c9d0a-193f-4e39-beb5-128f2eca3a6c-kube-api-access-sxlb7\") pod \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " Jul 02 23:32:09.993457 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.993414 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-envoy\") pod \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " Jul 02 23:32:09.993457 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.993432 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-workload-certs\") pod \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " Jul 02 23:32:09.993793 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.993472 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-workload-socket\") pod \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " Jul 02 23:32:09.993793 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.993511 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-data\") pod \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\" (UID: \"933c9d0a-193f-4e39-beb5-128f2eca3a6c\") " Jul 02 23:32:09.993793 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.993664 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "933c9d0a-193f-4e39-beb5-128f2eca3a6c" (UID: "933c9d0a-193f-4e39-beb5-128f2eca3a6c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:32:09.993793 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.993773 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-credential-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:09.994042 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.993911 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "933c9d0a-193f-4e39-beb5-128f2eca3a6c" (UID: "933c9d0a-193f-4e39-beb5-128f2eca3a6c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:32:09.994042 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.993951 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "933c9d0a-193f-4e39-beb5-128f2eca3a6c" (UID: "933c9d0a-193f-4e39-beb5-128f2eca3a6c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:32:09.994196 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.994177 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "933c9d0a-193f-4e39-beb5-128f2eca3a6c" (UID: "933c9d0a-193f-4e39-beb5-128f2eca3a6c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:32:09.994405 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.994387 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-data" (OuterVolumeSpecName: "istio-data") pod "933c9d0a-193f-4e39-beb5-128f2eca3a6c" (UID: "933c9d0a-193f-4e39-beb5-128f2eca3a6c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:32:09.996489 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.996467 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "933c9d0a-193f-4e39-beb5-128f2eca3a6c" (UID: "933c9d0a-193f-4e39-beb5-128f2eca3a6c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:32:09.996591 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.996488 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/933c9d0a-193f-4e39-beb5-128f2eca3a6c-kube-api-access-sxlb7" (OuterVolumeSpecName: "kube-api-access-sxlb7") pod "933c9d0a-193f-4e39-beb5-128f2eca3a6c" (UID: "933c9d0a-193f-4e39-beb5-128f2eca3a6c"). InnerVolumeSpecName "kube-api-access-sxlb7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:32:09.996591 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.996547 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "933c9d0a-193f-4e39-beb5-128f2eca3a6c" (UID: "933c9d0a-193f-4e39-beb5-128f2eca3a6c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:32:09.996591 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:09.996565 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-token" (OuterVolumeSpecName: "istio-token") pod "933c9d0a-193f-4e39-beb5-128f2eca3a6c" (UID: "933c9d0a-193f-4e39-beb5-128f2eca3a6c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:32:10.035521 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.035486 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:10.095196 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.095158 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istiod-ca-cert\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:10.095196 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.095198 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-token\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:10.095432 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.095213 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxlb7\" (UniqueName: \"kubernetes.io/projected/933c9d0a-193f-4e39-beb5-128f2eca3a6c-kube-api-access-sxlb7\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:10.095432 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.095228 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-envoy\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:10.095432 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.095243 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-workload-certs\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:10.095432 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.095256 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-workload-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:10.095432 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.095268 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-data\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:10.095432 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.095284 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/933c9d0a-193f-4e39-beb5-128f2eca3a6c-istio-podinfo\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:10.177410 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.177381 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb"] Jul 02 23:32:10.180060 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:32:10.180028 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11ddb3de_5506_41ee_bf98_5809c7b6d8fb.slice/crio-d0d02d341742f854123d201f199330c3ff4abf5e083ef7875c07471d785425c4 WatchSource:0}: Error finding container d0d02d341742f854123d201f199330c3ff4abf5e083ef7875c07471d785425c4: Status 404 returned error can't find the container with id d0d02d341742f854123d201f199330c3ff4abf5e083ef7875c07471d785425c4 Jul 02 23:32:10.182603 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.182573 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:32:10.182676 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.182638 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:32:10.182676 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.182669 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:32:10.539693 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.539599 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" event={"ID":"11ddb3de-5506-41ee-bf98-5809c7b6d8fb","Type":"ContainerStarted","Data":"ee7ec51da9776e0804406aeb4a873b23e7de7ae103b9927a7e68d2694c16d970"} Jul 02 23:32:10.539693 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.539660 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" event={"ID":"11ddb3de-5506-41ee-bf98-5809c7b6d8fb","Type":"ContainerStarted","Data":"d0d02d341742f854123d201f199330c3ff4abf5e083ef7875c07471d785425c4"} Jul 02 23:32:10.541126 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.541095 2583 generic.go:356] "Generic (PLEG): container finished" podID="933c9d0a-193f-4e39-beb5-128f2eca3a6c" containerID="353ed944b103a036830516b821cb6bc723da601906457da3c82d1ba778055b08" exitCode=0 Jul 02 23:32:10.541251 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.541139 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" Jul 02 23:32:10.541251 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.541174 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" event={"ID":"933c9d0a-193f-4e39-beb5-128f2eca3a6c","Type":"ContainerDied","Data":"353ed944b103a036830516b821cb6bc723da601906457da3c82d1ba778055b08"} Jul 02 23:32:10.541251 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.541233 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9" event={"ID":"933c9d0a-193f-4e39-beb5-128f2eca3a6c","Type":"ContainerDied","Data":"6a876493cd8c1656780bf36129c1e61f4bfc6b98123fcbb815eb591cf1d98035"} Jul 02 23:32:10.541368 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.541262 2583 scope.go:117] "RemoveContainer" containerID="353ed944b103a036830516b821cb6bc723da601906457da3c82d1ba778055b08" Jul 02 23:32:10.550916 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.550878 2583 scope.go:117] "RemoveContainer" containerID="353ed944b103a036830516b821cb6bc723da601906457da3c82d1ba778055b08" Jul 02 23:32:10.551212 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:32:10.551189 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"353ed944b103a036830516b821cb6bc723da601906457da3c82d1ba778055b08\": container with ID starting with 353ed944b103a036830516b821cb6bc723da601906457da3c82d1ba778055b08 not found: ID does not exist" containerID="353ed944b103a036830516b821cb6bc723da601906457da3c82d1ba778055b08" Jul 02 23:32:10.551275 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.551225 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"353ed944b103a036830516b821cb6bc723da601906457da3c82d1ba778055b08"} err="failed to get container status \"353ed944b103a036830516b821cb6bc723da601906457da3c82d1ba778055b08\": rpc error: code = NotFound desc = could not find container \"353ed944b103a036830516b821cb6bc723da601906457da3c82d1ba778055b08\": container with ID starting with 353ed944b103a036830516b821cb6bc723da601906457da3c82d1ba778055b08 not found: ID does not exist" Jul 02 23:32:10.557342 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.557296 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" podStartSLOduration=1.557277643 podStartE2EDuration="1.557277643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:32:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:32:10.55622384 +0000 UTC m=+1494.579041964" watchObservedRunningTime="2026-07-02 23:32:10.557277643 +0000 UTC m=+1494.580095759" Jul 02 23:32:10.568932 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.568905 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9"] Jul 02 23:32:10.574928 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:10.574902 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4f93315a-openshift-default-575bdf6c75-6cfj9"] Jul 02 23:32:11.036424 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:11.036395 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:11.041168 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:11.041136 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:11.547103 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:11.547075 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:11.548160 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:11.548142 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:12.537576 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:12.537537 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="933c9d0a-193f-4e39-beb5-128f2eca3a6c" path="/var/lib/kubelet/pods/933c9d0a-193f-4e39-beb5-128f2eca3a6c/volumes" Jul 02 23:32:22.420902 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:22.420850 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb"] Jul 02 23:32:22.421287 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:22.421118 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" podUID="11ddb3de-5506-41ee-bf98-5809c7b6d8fb" containerName="istio-proxy" containerID="cri-o://ee7ec51da9776e0804406aeb4a873b23e7de7ae103b9927a7e68d2694c16d970" gracePeriod=30 Jul 02 23:32:26.549870 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:26.549834 2583 patch_prober.go:28] interesting pod/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 02 23:32:26.550309 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:26.549951 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" podUID="11ddb3de-5506-41ee-bf98-5809c7b6d8fb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 23:32:27.625309 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.625276 2583 generic.go:356] "Generic (PLEG): container finished" podID="11ddb3de-5506-41ee-bf98-5809c7b6d8fb" containerID="ee7ec51da9776e0804406aeb4a873b23e7de7ae103b9927a7e68d2694c16d970" exitCode=0 Jul 02 23:32:27.625663 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.625331 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" event={"ID":"11ddb3de-5506-41ee-bf98-5809c7b6d8fb","Type":"ContainerDied","Data":"ee7ec51da9776e0804406aeb4a873b23e7de7ae103b9927a7e68d2694c16d970"} Jul 02 23:32:27.673347 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.673321 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:27.754067 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.753964 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-workload-socket\") pod \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " Jul 02 23:32:27.754067 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.754008 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-envoy\") pod \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " Jul 02 23:32:27.754067 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.754027 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wncc\" (UniqueName: \"kubernetes.io/projected/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-kube-api-access-9wncc\") pod \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " Jul 02 23:32:27.754378 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.754073 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-data\") pod \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " Jul 02 23:32:27.754378 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.754121 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-podinfo\") pod \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " Jul 02 23:32:27.754378 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.754149 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istiod-ca-cert\") pod \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " Jul 02 23:32:27.754378 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.754189 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-token\") pod \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " Jul 02 23:32:27.754378 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.754213 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-credential-socket\") pod \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " Jul 02 23:32:27.754378 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.754255 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-workload-certs\") pod \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\" (UID: \"11ddb3de-5506-41ee-bf98-5809c7b6d8fb\") " Jul 02 23:32:27.754378 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.754301 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "11ddb3de-5506-41ee-bf98-5809c7b6d8fb" (UID: "11ddb3de-5506-41ee-bf98-5809c7b6d8fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:32:27.754729 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.754424 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-data" (OuterVolumeSpecName: "istio-data") pod "11ddb3de-5506-41ee-bf98-5809c7b6d8fb" (UID: "11ddb3de-5506-41ee-bf98-5809c7b6d8fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:32:27.754729 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.754608 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "11ddb3de-5506-41ee-bf98-5809c7b6d8fb" (UID: "11ddb3de-5506-41ee-bf98-5809c7b6d8fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:32:27.754729 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.754616 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-workload-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:27.754729 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.754637 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-data\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:27.754729 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.754656 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "11ddb3de-5506-41ee-bf98-5809c7b6d8fb" (UID: "11ddb3de-5506-41ee-bf98-5809c7b6d8fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:32:27.754973 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.754864 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "11ddb3de-5506-41ee-bf98-5809c7b6d8fb" (UID: "11ddb3de-5506-41ee-bf98-5809c7b6d8fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:32:27.756569 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.756543 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "11ddb3de-5506-41ee-bf98-5809c7b6d8fb" (UID: "11ddb3de-5506-41ee-bf98-5809c7b6d8fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:32:27.756569 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.756550 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-kube-api-access-9wncc" (OuterVolumeSpecName: "kube-api-access-9wncc") pod "11ddb3de-5506-41ee-bf98-5809c7b6d8fb" (UID: "11ddb3de-5506-41ee-bf98-5809c7b6d8fb"). InnerVolumeSpecName "kube-api-access-9wncc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:32:27.756761 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.756746 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "11ddb3de-5506-41ee-bf98-5809c7b6d8fb" (UID: "11ddb3de-5506-41ee-bf98-5809c7b6d8fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:32:27.756949 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.756929 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-token" (OuterVolumeSpecName: "istio-token") pod "11ddb3de-5506-41ee-bf98-5809c7b6d8fb" (UID: "11ddb3de-5506-41ee-bf98-5809c7b6d8fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:32:27.855964 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.855871 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-token\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:27.855964 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.855957 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-credential-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:27.855964 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.855972 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-workload-certs\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:27.856265 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.855985 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-envoy\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:27.856265 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.855998 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wncc\" (UniqueName: \"kubernetes.io/projected/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-kube-api-access-9wncc\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:27.856265 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.856009 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istio-podinfo\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:27.856265 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:27.856020 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11ddb3de-5506-41ee-bf98-5809c7b6d8fb-istiod-ca-cert\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:28.324971 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.324930 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr"] Jul 02 23:32:28.325531 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.325509 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="933c9d0a-193f-4e39-beb5-128f2eca3a6c" containerName="istio-proxy" Jul 02 23:32:28.325613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.325535 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="933c9d0a-193f-4e39-beb5-128f2eca3a6c" containerName="istio-proxy" Jul 02 23:32:28.325613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.325548 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11ddb3de-5506-41ee-bf98-5809c7b6d8fb" containerName="istio-proxy" Jul 02 23:32:28.325613 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.325557 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="11ddb3de-5506-41ee-bf98-5809c7b6d8fb" containerName="istio-proxy" Jul 02 23:32:28.325709 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.325700 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="933c9d0a-193f-4e39-beb5-128f2eca3a6c" containerName="istio-proxy" Jul 02 23:32:28.325742 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.325714 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="11ddb3de-5506-41ee-bf98-5809c7b6d8fb" containerName="istio-proxy" Jul 02 23:32:28.329610 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.329587 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.331939 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.331916 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-4d528601-openshift-default-dockercfg-9k5gx\"" Jul 02 23:32:28.340505 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.340477 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr"] Jul 02 23:32:28.361281 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.361246 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-workload-socket\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.361281 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.361282 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlrgc\" (UniqueName: \"kubernetes.io/projected/40b4c987-774b-49f1-9930-7ba9fedb42ce-kube-api-access-jlrgc\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.361497 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.361314 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-workload-certs\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.361497 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.361346 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40b4c987-774b-49f1-9930-7ba9fedb42ce-istiod-ca-cert\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.361497 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.361374 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-data\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.361497 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.361394 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-envoy\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.361497 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.361410 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-podinfo\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.361497 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.361445 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-token\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.361497 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.361474 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-credential-socket\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.462505 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.462463 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-credential-socket\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.462505 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.462504 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-workload-socket\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.462728 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.462527 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlrgc\" (UniqueName: \"kubernetes.io/projected/40b4c987-774b-49f1-9930-7ba9fedb42ce-kube-api-access-jlrgc\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.462728 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.462572 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-workload-certs\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.462728 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.462594 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40b4c987-774b-49f1-9930-7ba9fedb42ce-istiod-ca-cert\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.462728 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.462642 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-data\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.462728 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.462675 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-envoy\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.462728 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.462699 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-podinfo\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.463094 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.462775 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-token\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.463094 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.462982 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-workload-socket\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.463094 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.462997 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-workload-certs\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.463229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.463121 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-data\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.463229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.463173 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-credential-socket\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.463387 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.463368 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40b4c987-774b-49f1-9930-7ba9fedb42ce-istiod-ca-cert\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.465060 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.465040 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-envoy\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.465300 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.465283 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-podinfo\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.471829 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.471801 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-token\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.472010 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.471991 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlrgc\" (UniqueName: \"kubernetes.io/projected/40b4c987-774b-49f1-9930-7ba9fedb42ce-kube-api-access-jlrgc\") pod \"e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.631719 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.631628 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" event={"ID":"11ddb3de-5506-41ee-bf98-5809c7b6d8fb","Type":"ContainerDied","Data":"d0d02d341742f854123d201f199330c3ff4abf5e083ef7875c07471d785425c4"} Jul 02 23:32:28.631719 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.631652 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb" Jul 02 23:32:28.631719 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.631691 2583 scope.go:117] "RemoveContainer" containerID="ee7ec51da9776e0804406aeb4a873b23e7de7ae103b9927a7e68d2694c16d970" Jul 02 23:32:28.642013 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.641994 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:28.646870 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.646849 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb"] Jul 02 23:32:28.650523 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.650503 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-7dd7cae8-openshift-default-85599bfc65-4tvvb"] Jul 02 23:32:28.785439 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.785414 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr"] Jul 02 23:32:28.787718 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:32:28.787688 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b4c987_774b_49f1_9930_7ba9fedb42ce.slice/crio-421f35ce3f636151b06ea0384c27d79bffb32677e12363001a74d94485fe658e WatchSource:0}: Error finding container 421f35ce3f636151b06ea0384c27d79bffb32677e12363001a74d94485fe658e: Status 404 returned error can't find the container with id 421f35ce3f636151b06ea0384c27d79bffb32677e12363001a74d94485fe658e Jul 02 23:32:28.790085 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.790050 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:32:28.790156 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.790140 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:32:28.790190 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:28.790180 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:32:29.639141 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:29.639098 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" event={"ID":"40b4c987-774b-49f1-9930-7ba9fedb42ce","Type":"ContainerStarted","Data":"c48575a686f7a27de2fb9e7f4170c0b23ae86e3c93c762f8637ef6888f320952"} Jul 02 23:32:29.639541 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:29.639160 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" event={"ID":"40b4c987-774b-49f1-9930-7ba9fedb42ce","Type":"ContainerStarted","Data":"421f35ce3f636151b06ea0384c27d79bffb32677e12363001a74d94485fe658e"} Jul 02 23:32:29.642225 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:29.642197 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:29.646918 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:29.646878 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:29.665547 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:29.665455 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" podStartSLOduration=1.665434149 podStartE2EDuration="1.665434149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:32:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:32:29.663825464 +0000 UTC m=+1513.686643578" watchObservedRunningTime="2026-07-02 23:32:29.665434149 +0000 UTC m=+1513.688252263" Jul 02 23:32:30.532502 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:30.532464 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11ddb3de-5506-41ee-bf98-5809c7b6d8fb" path="/var/lib/kubelet/pods/11ddb3de-5506-41ee-bf98-5809c7b6d8fb/volumes" Jul 02 23:32:30.645705 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:30.645674 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:30.646711 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:30.646692 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:34.145640 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:34.145606 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr"] Jul 02 23:32:34.146069 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:34.145824 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" podUID="40b4c987-774b-49f1-9930-7ba9fedb42ce" containerName="istio-proxy" containerID="cri-o://c48575a686f7a27de2fb9e7f4170c0b23ae86e3c93c762f8637ef6888f320952" gracePeriod=30 Jul 02 23:32:40.203416 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.203387 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:40.278486 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.278380 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlrgc\" (UniqueName: \"kubernetes.io/projected/40b4c987-774b-49f1-9930-7ba9fedb42ce-kube-api-access-jlrgc\") pod \"40b4c987-774b-49f1-9930-7ba9fedb42ce\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " Jul 02 23:32:40.278486 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.278465 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-podinfo\") pod \"40b4c987-774b-49f1-9930-7ba9fedb42ce\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " Jul 02 23:32:40.278486 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.278486 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40b4c987-774b-49f1-9930-7ba9fedb42ce-istiod-ca-cert\") pod \"40b4c987-774b-49f1-9930-7ba9fedb42ce\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " Jul 02 23:32:40.278737 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.278500 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-token\") pod \"40b4c987-774b-49f1-9930-7ba9fedb42ce\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " Jul 02 23:32:40.278737 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.278542 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-credential-socket\") pod \"40b4c987-774b-49f1-9930-7ba9fedb42ce\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " Jul 02 23:32:40.278737 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.278592 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-workload-socket\") pod \"40b4c987-774b-49f1-9930-7ba9fedb42ce\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " Jul 02 23:32:40.278737 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.278607 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-workload-certs\") pod \"40b4c987-774b-49f1-9930-7ba9fedb42ce\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " Jul 02 23:32:40.278737 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.278637 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-data\") pod \"40b4c987-774b-49f1-9930-7ba9fedb42ce\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " Jul 02 23:32:40.278737 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.278652 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-envoy\") pod \"40b4c987-774b-49f1-9930-7ba9fedb42ce\" (UID: \"40b4c987-774b-49f1-9930-7ba9fedb42ce\") " Jul 02 23:32:40.280297 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.280256 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "40b4c987-774b-49f1-9930-7ba9fedb42ce" (UID: "40b4c987-774b-49f1-9930-7ba9fedb42ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:32:40.280297 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.280269 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "40b4c987-774b-49f1-9930-7ba9fedb42ce" (UID: "40b4c987-774b-49f1-9930-7ba9fedb42ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:32:40.280487 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.280361 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40b4c987-774b-49f1-9930-7ba9fedb42ce-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "40b4c987-774b-49f1-9930-7ba9fedb42ce" (UID: "40b4c987-774b-49f1-9930-7ba9fedb42ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:32:40.280487 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.280399 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "40b4c987-774b-49f1-9930-7ba9fedb42ce" (UID: "40b4c987-774b-49f1-9930-7ba9fedb42ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:32:40.280926 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.280861 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-data" (OuterVolumeSpecName: "istio-data") pod "40b4c987-774b-49f1-9930-7ba9fedb42ce" (UID: "40b4c987-774b-49f1-9930-7ba9fedb42ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:32:40.282240 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.282209 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "40b4c987-774b-49f1-9930-7ba9fedb42ce" (UID: "40b4c987-774b-49f1-9930-7ba9fedb42ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:32:40.282351 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.282291 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40b4c987-774b-49f1-9930-7ba9fedb42ce-kube-api-access-jlrgc" (OuterVolumeSpecName: "kube-api-access-jlrgc") pod "40b4c987-774b-49f1-9930-7ba9fedb42ce" (UID: "40b4c987-774b-49f1-9930-7ba9fedb42ce"). InnerVolumeSpecName "kube-api-access-jlrgc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:32:40.282610 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.282588 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "40b4c987-774b-49f1-9930-7ba9fedb42ce" (UID: "40b4c987-774b-49f1-9930-7ba9fedb42ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:32:40.282661 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.282640 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-token" (OuterVolumeSpecName: "istio-token") pod "40b4c987-774b-49f1-9930-7ba9fedb42ce" (UID: "40b4c987-774b-49f1-9930-7ba9fedb42ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:32:40.379695 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.379654 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-workload-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:40.379695 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.379691 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-workload-certs\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:40.379695 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.379701 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-data\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:40.379967 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.379708 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-envoy\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:40.379967 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.379717 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlrgc\" (UniqueName: \"kubernetes.io/projected/40b4c987-774b-49f1-9930-7ba9fedb42ce-kube-api-access-jlrgc\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:40.379967 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.379727 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-podinfo\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:40.379967 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.379735 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40b4c987-774b-49f1-9930-7ba9fedb42ce-istiod-ca-cert\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:40.379967 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.379743 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40b4c987-774b-49f1-9930-7ba9fedb42ce-istio-token\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:40.379967 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.379752 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b4c987-774b-49f1-9930-7ba9fedb42ce-credential-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:32:40.695513 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.695475 2583 generic.go:356] "Generic (PLEG): container finished" podID="40b4c987-774b-49f1-9930-7ba9fedb42ce" containerID="c48575a686f7a27de2fb9e7f4170c0b23ae86e3c93c762f8637ef6888f320952" exitCode=0 Jul 02 23:32:40.695678 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.695527 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" Jul 02 23:32:40.695678 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.695567 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" event={"ID":"40b4c987-774b-49f1-9930-7ba9fedb42ce","Type":"ContainerDied","Data":"c48575a686f7a27de2fb9e7f4170c0b23ae86e3c93c762f8637ef6888f320952"} Jul 02 23:32:40.695678 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.695627 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr" event={"ID":"40b4c987-774b-49f1-9930-7ba9fedb42ce","Type":"ContainerDied","Data":"421f35ce3f636151b06ea0384c27d79bffb32677e12363001a74d94485fe658e"} Jul 02 23:32:40.695678 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.695652 2583 scope.go:117] "RemoveContainer" containerID="c48575a686f7a27de2fb9e7f4170c0b23ae86e3c93c762f8637ef6888f320952" Jul 02 23:32:40.704357 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.704336 2583 scope.go:117] "RemoveContainer" containerID="c48575a686f7a27de2fb9e7f4170c0b23ae86e3c93c762f8637ef6888f320952" Jul 02 23:32:40.704623 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:32:40.704601 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c48575a686f7a27de2fb9e7f4170c0b23ae86e3c93c762f8637ef6888f320952\": container with ID starting with c48575a686f7a27de2fb9e7f4170c0b23ae86e3c93c762f8637ef6888f320952 not found: ID does not exist" containerID="c48575a686f7a27de2fb9e7f4170c0b23ae86e3c93c762f8637ef6888f320952" Jul 02 23:32:40.704668 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.704632 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c48575a686f7a27de2fb9e7f4170c0b23ae86e3c93c762f8637ef6888f320952"} err="failed to get container status \"c48575a686f7a27de2fb9e7f4170c0b23ae86e3c93c762f8637ef6888f320952\": rpc error: code = NotFound desc = could not find container \"c48575a686f7a27de2fb9e7f4170c0b23ae86e3c93c762f8637ef6888f320952\": container with ID starting with c48575a686f7a27de2fb9e7f4170c0b23ae86e3c93c762f8637ef6888f320952 not found: ID does not exist" Jul 02 23:32:40.711582 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.711555 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr"] Jul 02 23:32:40.713315 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:40.713294 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-4d528601-openshift-default-6bb4d997dc-ksbhr"] Jul 02 23:32:42.532715 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:32:42.532680 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40b4c987-774b-49f1-9930-7ba9fedb42ce" path="/var/lib/kubelet/pods/40b4c987-774b-49f1-9930-7ba9fedb42ce/volumes" Jul 02 23:34:14.410884 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.410798 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk"] Jul 02 23:34:14.411383 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.411266 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40b4c987-774b-49f1-9930-7ba9fedb42ce" containerName="istio-proxy" Jul 02 23:34:14.411383 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.411283 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="40b4c987-774b-49f1-9930-7ba9fedb42ce" containerName="istio-proxy" Jul 02 23:34:14.411383 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.411361 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="40b4c987-774b-49f1-9930-7ba9fedb42ce" containerName="istio-proxy" Jul 02 23:34:14.414540 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.414521 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.417037 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.417017 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ceefe56c-openshift-default-dockercfg-pzdfg\"" Jul 02 23:34:14.436010 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.435973 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk"] Jul 02 23:34:14.501192 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.501146 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istiod-ca-cert\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.501382 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.501269 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-token\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.501382 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.501332 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-workload-socket\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.501382 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.501360 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-envoy\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.501560 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.501392 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-podinfo\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.501560 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.501469 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-workload-certs\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.501560 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.501515 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72j8q\" (UniqueName: \"kubernetes.io/projected/d8d2eed0-cdd2-4def-95cd-a42bc4516028-kube-api-access-72j8q\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.501560 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.501552 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-credential-socket\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.501728 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.501665 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-data\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.602607 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.602565 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-workload-socket\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.602607 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.602611 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-envoy\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.602982 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.602633 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-podinfo\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.602982 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.602677 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-workload-certs\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.602982 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.602710 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72j8q\" (UniqueName: \"kubernetes.io/projected/d8d2eed0-cdd2-4def-95cd-a42bc4516028-kube-api-access-72j8q\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.602982 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.602740 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-credential-socket\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.602982 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.602784 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-data\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.603233 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.602962 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istiod-ca-cert\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.603233 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.603073 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-workload-socket\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.603233 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.603085 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-token\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.603564 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.603542 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-credential-socket\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.603810 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.603789 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-workload-certs\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.603924 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.603877 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-data\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.603973 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.603946 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istiod-ca-cert\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.605252 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.605222 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-envoy\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.605376 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.605359 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-podinfo\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.610568 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.610546 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-token\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.610682 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.610649 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72j8q\" (UniqueName: \"kubernetes.io/projected/d8d2eed0-cdd2-4def-95cd-a42bc4516028-kube-api-access-72j8q\") pod \"e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.725165 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.725068 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:14.870602 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.870573 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk"] Jul 02 23:34:14.872980 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:34:14.872932 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8d2eed0_cdd2_4def_95cd_a42bc4516028.slice/crio-e19c8402834c74a5a0fe958795aa72400c7e941d5cc901e563f9abae1a694130 WatchSource:0}: Error finding container e19c8402834c74a5a0fe958795aa72400c7e941d5cc901e563f9abae1a694130: Status 404 returned error can't find the container with id e19c8402834c74a5a0fe958795aa72400c7e941d5cc901e563f9abae1a694130 Jul 02 23:34:14.875384 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.875339 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:34:14.875478 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.875434 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:34:14.875517 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:14.875474 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:34:15.152043 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:15.152004 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" event={"ID":"d8d2eed0-cdd2-4def-95cd-a42bc4516028","Type":"ContainerStarted","Data":"2ed8bf0456d97e0f7f66101fda73a0980a23cf88f15ca852e6813368067ff099"} Jul 02 23:34:15.152254 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:15.152058 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" event={"ID":"d8d2eed0-cdd2-4def-95cd-a42bc4516028","Type":"ContainerStarted","Data":"e19c8402834c74a5a0fe958795aa72400c7e941d5cc901e563f9abae1a694130"} Jul 02 23:34:15.171388 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:15.171314 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" podStartSLOduration=1.171295115 podStartE2EDuration="1.171295115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:34:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:34:15.170772609 +0000 UTC m=+1619.193590720" watchObservedRunningTime="2026-07-02 23:34:15.171295115 +0000 UTC m=+1619.194113229" Jul 02 23:34:15.725677 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:15.725643 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:15.730406 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:15.730377 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:16.158146 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:16.158111 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:16.159034 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:16.159012 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:31.160055 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:31.159955 2583 patch_prober.go:28] interesting pod/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 02 23:34:31.160055 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:31.160022 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" podUID="d8d2eed0-cdd2-4def-95cd-a42bc4516028" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 02 23:34:31.242447 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:31.242414 2583 generic.go:356] "Generic (PLEG): container finished" podID="d8d2eed0-cdd2-4def-95cd-a42bc4516028" containerID="2ed8bf0456d97e0f7f66101fda73a0980a23cf88f15ca852e6813368067ff099" exitCode=0 Jul 02 23:34:31.242630 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:31.242492 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" event={"ID":"d8d2eed0-cdd2-4def-95cd-a42bc4516028","Type":"ContainerDied","Data":"2ed8bf0456d97e0f7f66101fda73a0980a23cf88f15ca852e6813368067ff099"} Jul 02 23:34:31.242910 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:31.242882 2583 scope.go:117] "RemoveContainer" containerID="2ed8bf0456d97e0f7f66101fda73a0980a23cf88f15ca852e6813368067ff099" Jul 02 23:34:31.243921 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:31.243873 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:34:31.243990 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:31.243964 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:34:31.244027 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:31.244014 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:34:32.249035 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:32.249002 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" event={"ID":"d8d2eed0-cdd2-4def-95cd-a42bc4516028","Type":"ContainerStarted","Data":"d08d70936a1c94e9a70a5cd3f7b3656af7ef4fa69903030c2e6892286716c204"} Jul 02 23:34:32.725566 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:32.725522 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:32.725773 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:32.725747 2583 patch_prober.go:28] interesting pod/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 02 23:34:32.725838 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:32.725802 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" podUID="d8d2eed0-cdd2-4def-95cd-a42bc4516028" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 02 23:34:33.726033 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:33.725996 2583 patch_prober.go:28] interesting pod/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 02 23:34:33.726530 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:33.726058 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" podUID="d8d2eed0-cdd2-4def-95cd-a42bc4516028" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 02 23:34:34.725776 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:34.725738 2583 patch_prober.go:28] interesting pod/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 02 23:34:34.725993 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:34.725798 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" podUID="d8d2eed0-cdd2-4def-95cd-a42bc4516028" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 02 23:34:35.144043 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:35.143992 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk"] Jul 02 23:34:35.144861 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:35.144825 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" podUID="d8d2eed0-cdd2-4def-95cd-a42bc4516028" containerName="istio-proxy" containerID="cri-o://d08d70936a1c94e9a70a5cd3f7b3656af7ef4fa69903030c2e6892286716c204" gracePeriod=30 Jul 02 23:34:41.292482 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.292444 2583 generic.go:356] "Generic (PLEG): container finished" podID="d8d2eed0-cdd2-4def-95cd-a42bc4516028" containerID="d08d70936a1c94e9a70a5cd3f7b3656af7ef4fa69903030c2e6892286716c204" exitCode=0 Jul 02 23:34:41.292850 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.292489 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" event={"ID":"d8d2eed0-cdd2-4def-95cd-a42bc4516028","Type":"ContainerDied","Data":"d08d70936a1c94e9a70a5cd3f7b3656af7ef4fa69903030c2e6892286716c204"} Jul 02 23:34:41.292850 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.292535 2583 scope.go:117] "RemoveContainer" containerID="2ed8bf0456d97e0f7f66101fda73a0980a23cf88f15ca852e6813368067ff099" Jul 02 23:34:41.399815 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.399785 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:41.559689 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.559655 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72j8q\" (UniqueName: \"kubernetes.io/projected/d8d2eed0-cdd2-4def-95cd-a42bc4516028-kube-api-access-72j8q\") pod \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " Jul 02 23:34:41.559872 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.559730 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-credential-socket\") pod \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " Jul 02 23:34:41.559872 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.559760 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-workload-socket\") pod \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " Jul 02 23:34:41.559872 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.559791 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-data\") pod \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " Jul 02 23:34:41.559872 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.559811 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-podinfo\") pod \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " Jul 02 23:34:41.559872 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.559830 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-workload-certs\") pod \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " Jul 02 23:34:41.559872 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.559858 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-envoy\") pod \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " Jul 02 23:34:41.560209 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.559917 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istiod-ca-cert\") pod \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " Jul 02 23:34:41.560209 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.559961 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-token\") pod \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\" (UID: \"d8d2eed0-cdd2-4def-95cd-a42bc4516028\") " Jul 02 23:34:41.560209 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.560067 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d8d2eed0-cdd2-4def-95cd-a42bc4516028" (UID: "d8d2eed0-cdd2-4def-95cd-a42bc4516028"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:34:41.560209 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.560196 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d8d2eed0-cdd2-4def-95cd-a42bc4516028" (UID: "d8d2eed0-cdd2-4def-95cd-a42bc4516028"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:34:41.560429 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.560295 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-workload-certs\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:34:41.560429 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.560315 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-credential-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:34:41.560525 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.560491 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d8d2eed0-cdd2-4def-95cd-a42bc4516028" (UID: "d8d2eed0-cdd2-4def-95cd-a42bc4516028"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:34:41.560583 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.560539 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d8d2eed0-cdd2-4def-95cd-a42bc4516028" (UID: "d8d2eed0-cdd2-4def-95cd-a42bc4516028"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:34:41.560801 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.560683 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-data" (OuterVolumeSpecName: "istio-data") pod "d8d2eed0-cdd2-4def-95cd-a42bc4516028" (UID: "d8d2eed0-cdd2-4def-95cd-a42bc4516028"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:34:41.562237 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.562210 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d8d2eed0-cdd2-4def-95cd-a42bc4516028" (UID: "d8d2eed0-cdd2-4def-95cd-a42bc4516028"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:34:41.562342 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.562241 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-token" (OuterVolumeSpecName: "istio-token") pod "d8d2eed0-cdd2-4def-95cd-a42bc4516028" (UID: "d8d2eed0-cdd2-4def-95cd-a42bc4516028"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:34:41.562614 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.562591 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8d2eed0-cdd2-4def-95cd-a42bc4516028-kube-api-access-72j8q" (OuterVolumeSpecName: "kube-api-access-72j8q") pod "d8d2eed0-cdd2-4def-95cd-a42bc4516028" (UID: "d8d2eed0-cdd2-4def-95cd-a42bc4516028"). InnerVolumeSpecName "kube-api-access-72j8q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:34:41.562744 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.562661 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d8d2eed0-cdd2-4def-95cd-a42bc4516028" (UID: "d8d2eed0-cdd2-4def-95cd-a42bc4516028"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:34:41.661308 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.661259 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72j8q\" (UniqueName: \"kubernetes.io/projected/d8d2eed0-cdd2-4def-95cd-a42bc4516028-kube-api-access-72j8q\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:34:41.661308 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.661301 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-workload-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:34:41.661308 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.661311 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-data\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:34:41.661308 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.661321 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-podinfo\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:34:41.661594 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.661329 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-envoy\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:34:41.661594 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.661338 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istiod-ca-cert\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:34:41.661594 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:41.661345 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8d2eed0-cdd2-4def-95cd-a42bc4516028-istio-token\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:34:42.298819 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:42.298787 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" Jul 02 23:34:42.299290 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:42.298821 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk" event={"ID":"d8d2eed0-cdd2-4def-95cd-a42bc4516028","Type":"ContainerDied","Data":"e19c8402834c74a5a0fe958795aa72400c7e941d5cc901e563f9abae1a694130"} Jul 02 23:34:42.299290 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:42.298877 2583 scope.go:117] "RemoveContainer" containerID="d08d70936a1c94e9a70a5cd3f7b3656af7ef4fa69903030c2e6892286716c204" Jul 02 23:34:42.327224 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:42.327192 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk"] Jul 02 23:34:42.331415 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:42.331389 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ceefe56c-openshift-default-5dfd5cb7f5-hrctk"] Jul 02 23:34:42.532948 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:42.532881 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8d2eed0-cdd2-4def-95cd-a42bc4516028" path="/var/lib/kubelet/pods/d8d2eed0-cdd2-4def-95cd-a42bc4516028/volumes" Jul 02 23:34:53.994824 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:53.994788 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86"] Jul 02 23:34:53.995344 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:53.995185 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8d2eed0-cdd2-4def-95cd-a42bc4516028" containerName="istio-proxy" Jul 02 23:34:53.995344 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:53.995199 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8d2eed0-cdd2-4def-95cd-a42bc4516028" containerName="istio-proxy" Jul 02 23:34:53.995344 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:53.995210 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8d2eed0-cdd2-4def-95cd-a42bc4516028" containerName="istio-proxy" Jul 02 23:34:53.995344 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:53.995216 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8d2eed0-cdd2-4def-95cd-a42bc4516028" containerName="istio-proxy" Jul 02 23:34:53.995344 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:53.995273 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8d2eed0-cdd2-4def-95cd-a42bc4516028" containerName="istio-proxy" Jul 02 23:34:53.995344 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:53.995282 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8d2eed0-cdd2-4def-95cd-a42bc4516028" containerName="istio-proxy" Jul 02 23:34:53.997220 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:53.997198 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:53.999646 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:53.999620 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4cfb54d6-openshift-default-dockercfg-rgswz\"" Jul 02 23:34:54.009539 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.009513 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86"] Jul 02 23:34:54.177534 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.177454 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-data\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.177806 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.177600 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-podinfo\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.177806 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.177687 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-workload-socket\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.177806 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.177717 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istiod-ca-cert\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.177806 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.177800 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-credential-socket\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.178104 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.177824 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-token\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.178104 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.177872 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l588r\" (UniqueName: \"kubernetes.io/projected/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-kube-api-access-l588r\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.178104 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.177935 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-envoy\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.178104 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.177973 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-workload-certs\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.278916 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.278809 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-workload-socket\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.278916 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.278848 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istiod-ca-cert\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.279164 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.278920 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-credential-socket\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.279164 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.278939 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-token\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.279164 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.278965 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l588r\" (UniqueName: \"kubernetes.io/projected/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-kube-api-access-l588r\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.279164 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.278987 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-envoy\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.279164 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.279012 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-workload-certs\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.279164 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.279050 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-data\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.279164 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.279107 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-podinfo\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.279677 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.279645 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-workload-socket\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.279801 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.279654 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istiod-ca-cert\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.279868 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.279803 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-workload-certs\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.279868 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.279834 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-credential-socket\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.280022 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.279919 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-data\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.281711 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.281678 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-envoy\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.282103 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.282085 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-podinfo\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.292148 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.292125 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-token\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.292648 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.292630 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l588r\" (UniqueName: \"kubernetes.io/projected/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-kube-api-access-l588r\") pod \"e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.308604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.308561 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:54.460285 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.460259 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86"] Jul 02 23:34:54.462315 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:34:54.462281 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda38864b7_f740_43e5_91bb_ec18fd3ec4ef.slice/crio-5f75418d55fea14b1dae542d5812a58a1cc261771acafedc9823e1092b79cc6d WatchSource:0}: Error finding container 5f75418d55fea14b1dae542d5812a58a1cc261771acafedc9823e1092b79cc6d: Status 404 returned error can't find the container with id 5f75418d55fea14b1dae542d5812a58a1cc261771acafedc9823e1092b79cc6d Jul 02 23:34:54.464437 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.464402 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:34:54.464508 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.464475 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:34:54.464508 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:54.464505 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:34:55.361724 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:55.361683 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" event={"ID":"a38864b7-f740-43e5-91bb-ec18fd3ec4ef","Type":"ContainerStarted","Data":"4d98aca6e2f77157e7e70d7edc9cade5e761a895dfc21af0f9a9c2cb721e237d"} Jul 02 23:34:55.362315 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:55.361742 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" event={"ID":"a38864b7-f740-43e5-91bb-ec18fd3ec4ef","Type":"ContainerStarted","Data":"5f75418d55fea14b1dae542d5812a58a1cc261771acafedc9823e1092b79cc6d"} Jul 02 23:34:55.380167 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:55.380110 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" podStartSLOduration=2.380095332 podStartE2EDuration="2.380095332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:34:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:34:55.379609772 +0000 UTC m=+1659.402427918" watchObservedRunningTime="2026-07-02 23:34:55.380095332 +0000 UTC m=+1659.402913458" Jul 02 23:34:56.309646 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:56.309600 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:56.314573 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:56.314546 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:56.366924 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:56.366882 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:34:56.368016 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:34:56.367997 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:35:10.438705 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:10.438668 2583 generic.go:356] "Generic (PLEG): container finished" podID="a38864b7-f740-43e5-91bb-ec18fd3ec4ef" containerID="4d98aca6e2f77157e7e70d7edc9cade5e761a895dfc21af0f9a9c2cb721e237d" exitCode=0 Jul 02 23:35:10.439341 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:10.438722 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" event={"ID":"a38864b7-f740-43e5-91bb-ec18fd3ec4ef","Type":"ContainerDied","Data":"4d98aca6e2f77157e7e70d7edc9cade5e761a895dfc21af0f9a9c2cb721e237d"} Jul 02 23:35:10.439341 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:10.439266 2583 scope.go:117] "RemoveContainer" containerID="4d98aca6e2f77157e7e70d7edc9cade5e761a895dfc21af0f9a9c2cb721e237d" Jul 02 23:35:10.440492 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:10.440428 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:35:10.440650 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:10.440524 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:35:10.440650 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:10.440571 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:35:11.367344 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:11.367285 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:35:11.445619 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:11.445585 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" event={"ID":"a38864b7-f740-43e5-91bb-ec18fd3ec4ef","Type":"ContainerStarted","Data":"28d89bd3542b4c0ca62031b711c8f25b22092830c2e62a6f91d8536eb35b130c"} Jul 02 23:35:12.308991 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:12.308957 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:35:12.309964 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:12.309935 2583 patch_prober.go:28] interesting pod/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 02 23:35:12.310093 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:12.309984 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" podUID="a38864b7-f740-43e5-91bb-ec18fd3ec4ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 02 23:35:13.309233 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:13.309195 2583 patch_prober.go:28] interesting pod/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 02 23:35:13.309608 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:13.309256 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" podUID="a38864b7-f740-43e5-91bb-ec18fd3ec4ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 02 23:35:14.309667 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:14.309626 2583 patch_prober.go:28] interesting pod/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 02 23:35:14.310125 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:14.309689 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" podUID="a38864b7-f740-43e5-91bb-ec18fd3ec4ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 02 23:35:15.309707 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:15.309664 2583 patch_prober.go:28] interesting pod/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 02 23:35:15.310124 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:15.309724 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" podUID="a38864b7-f740-43e5-91bb-ec18fd3ec4ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 02 23:35:16.313874 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:16.313843 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:35:16.314333 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:16.314181 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:35:16.315044 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:35:16.315023 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:36:32.866489 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:32.866449 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86"] Jul 02 23:36:32.867043 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:32.866712 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" podUID="a38864b7-f740-43e5-91bb-ec18fd3ec4ef" containerName="istio-proxy" containerID="cri-o://28d89bd3542b4c0ca62031b711c8f25b22092830c2e62a6f91d8536eb35b130c" gracePeriod=30 Jul 02 23:36:38.131099 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.131030 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:36:38.242327 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.242284 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istiod-ca-cert\") pod \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " Jul 02 23:36:38.242509 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.242346 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-podinfo\") pod \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " Jul 02 23:36:38.242509 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.242372 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-credential-socket\") pod \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " Jul 02 23:36:38.242509 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.242474 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-workload-certs\") pod \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " Jul 02 23:36:38.242679 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.242551 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-envoy\") pod \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " Jul 02 23:36:38.242679 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.242608 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-data\") pod \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " Jul 02 23:36:38.242679 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.242615 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a38864b7-f740-43e5-91bb-ec18fd3ec4ef" (UID: "a38864b7-f740-43e5-91bb-ec18fd3ec4ef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:36:38.242679 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.242631 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-workload-socket\") pod \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " Jul 02 23:36:38.242679 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.242662 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l588r\" (UniqueName: \"kubernetes.io/projected/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-kube-api-access-l588r\") pod \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " Jul 02 23:36:38.242679 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.242660 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a38864b7-f740-43e5-91bb-ec18fd3ec4ef" (UID: "a38864b7-f740-43e5-91bb-ec18fd3ec4ef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:36:38.243022 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.242700 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-token\") pod \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\" (UID: \"a38864b7-f740-43e5-91bb-ec18fd3ec4ef\") " Jul 02 23:36:38.243022 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.242743 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a38864b7-f740-43e5-91bb-ec18fd3ec4ef" (UID: "a38864b7-f740-43e5-91bb-ec18fd3ec4ef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:36:38.243022 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.242949 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a38864b7-f740-43e5-91bb-ec18fd3ec4ef" (UID: "a38864b7-f740-43e5-91bb-ec18fd3ec4ef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:36:38.243190 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.243040 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istiod-ca-cert\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:36:38.243190 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.243060 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-credential-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:36:38.243190 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.243073 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-workload-certs\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:36:38.243657 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.243631 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-data" (OuterVolumeSpecName: "istio-data") pod "a38864b7-f740-43e5-91bb-ec18fd3ec4ef" (UID: "a38864b7-f740-43e5-91bb-ec18fd3ec4ef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:36:38.245141 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.245100 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a38864b7-f740-43e5-91bb-ec18fd3ec4ef" (UID: "a38864b7-f740-43e5-91bb-ec18fd3ec4ef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:36:38.245248 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.245166 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-token" (OuterVolumeSpecName: "istio-token") pod "a38864b7-f740-43e5-91bb-ec18fd3ec4ef" (UID: "a38864b7-f740-43e5-91bb-ec18fd3ec4ef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:36:38.245248 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.245168 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-kube-api-access-l588r" (OuterVolumeSpecName: "kube-api-access-l588r") pod "a38864b7-f740-43e5-91bb-ec18fd3ec4ef" (UID: "a38864b7-f740-43e5-91bb-ec18fd3ec4ef"). InnerVolumeSpecName "kube-api-access-l588r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:36:38.245364 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.245251 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a38864b7-f740-43e5-91bb-ec18fd3ec4ef" (UID: "a38864b7-f740-43e5-91bb-ec18fd3ec4ef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:36:38.344140 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.344103 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-podinfo\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:36:38.344140 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.344133 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-envoy\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:36:38.344140 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.344143 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-data\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:36:38.344140 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.344150 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-workload-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:36:38.344414 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.344160 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l588r\" (UniqueName: \"kubernetes.io/projected/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-kube-api-access-l588r\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:36:38.344414 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.344168 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a38864b7-f740-43e5-91bb-ec18fd3ec4ef-istio-token\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:36:38.860757 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.860721 2583 generic.go:356] "Generic (PLEG): container finished" podID="a38864b7-f740-43e5-91bb-ec18fd3ec4ef" containerID="28d89bd3542b4c0ca62031b711c8f25b22092830c2e62a6f91d8536eb35b130c" exitCode=0 Jul 02 23:36:38.860981 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.860763 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" Jul 02 23:36:38.860981 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.860810 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" event={"ID":"a38864b7-f740-43e5-91bb-ec18fd3ec4ef","Type":"ContainerDied","Data":"28d89bd3542b4c0ca62031b711c8f25b22092830c2e62a6f91d8536eb35b130c"} Jul 02 23:36:38.860981 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.860870 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86" event={"ID":"a38864b7-f740-43e5-91bb-ec18fd3ec4ef","Type":"ContainerDied","Data":"5f75418d55fea14b1dae542d5812a58a1cc261771acafedc9823e1092b79cc6d"} Jul 02 23:36:38.860981 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.860925 2583 scope.go:117] "RemoveContainer" containerID="28d89bd3542b4c0ca62031b711c8f25b22092830c2e62a6f91d8536eb35b130c" Jul 02 23:36:38.870306 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.870282 2583 scope.go:117] "RemoveContainer" containerID="4d98aca6e2f77157e7e70d7edc9cade5e761a895dfc21af0f9a9c2cb721e237d" Jul 02 23:36:38.880534 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.880515 2583 scope.go:117] "RemoveContainer" containerID="28d89bd3542b4c0ca62031b711c8f25b22092830c2e62a6f91d8536eb35b130c" Jul 02 23:36:38.880950 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:36:38.880920 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28d89bd3542b4c0ca62031b711c8f25b22092830c2e62a6f91d8536eb35b130c\": container with ID starting with 28d89bd3542b4c0ca62031b711c8f25b22092830c2e62a6f91d8536eb35b130c not found: ID does not exist" containerID="28d89bd3542b4c0ca62031b711c8f25b22092830c2e62a6f91d8536eb35b130c" Jul 02 23:36:38.881012 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.880965 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28d89bd3542b4c0ca62031b711c8f25b22092830c2e62a6f91d8536eb35b130c"} err="failed to get container status \"28d89bd3542b4c0ca62031b711c8f25b22092830c2e62a6f91d8536eb35b130c\": rpc error: code = NotFound desc = could not find container \"28d89bd3542b4c0ca62031b711c8f25b22092830c2e62a6f91d8536eb35b130c\": container with ID starting with 28d89bd3542b4c0ca62031b711c8f25b22092830c2e62a6f91d8536eb35b130c not found: ID does not exist" Jul 02 23:36:38.881012 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.880991 2583 scope.go:117] "RemoveContainer" containerID="4d98aca6e2f77157e7e70d7edc9cade5e761a895dfc21af0f9a9c2cb721e237d" Jul 02 23:36:38.881299 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:36:38.881273 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d98aca6e2f77157e7e70d7edc9cade5e761a895dfc21af0f9a9c2cb721e237d\": container with ID starting with 4d98aca6e2f77157e7e70d7edc9cade5e761a895dfc21af0f9a9c2cb721e237d not found: ID does not exist" containerID="4d98aca6e2f77157e7e70d7edc9cade5e761a895dfc21af0f9a9c2cb721e237d" Jul 02 23:36:38.881358 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.881305 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d98aca6e2f77157e7e70d7edc9cade5e761a895dfc21af0f9a9c2cb721e237d"} err="failed to get container status \"4d98aca6e2f77157e7e70d7edc9cade5e761a895dfc21af0f9a9c2cb721e237d\": rpc error: code = NotFound desc = could not find container \"4d98aca6e2f77157e7e70d7edc9cade5e761a895dfc21af0f9a9c2cb721e237d\": container with ID starting with 4d98aca6e2f77157e7e70d7edc9cade5e761a895dfc21af0f9a9c2cb721e237d not found: ID does not exist" Jul 02 23:36:38.881358 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.881344 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86"] Jul 02 23:36:38.885478 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:38.885450 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4cfb54d6-openshift-default-668896d49f-pnx86"] Jul 02 23:36:40.532249 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:36:40.532208 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a38864b7-f740-43e5-91bb-ec18fd3ec4ef" path="/var/lib/kubelet/pods/a38864b7-f740-43e5-91bb-ec18fd3ec4ef/volumes" Jul 02 23:37:26.808541 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.808505 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4"] Jul 02 23:37:26.809055 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.808859 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a38864b7-f740-43e5-91bb-ec18fd3ec4ef" containerName="istio-proxy" Jul 02 23:37:26.809055 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.808870 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a38864b7-f740-43e5-91bb-ec18fd3ec4ef" containerName="istio-proxy" Jul 02 23:37:26.809055 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.808880 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a38864b7-f740-43e5-91bb-ec18fd3ec4ef" containerName="istio-proxy" Jul 02 23:37:26.809055 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.808904 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a38864b7-f740-43e5-91bb-ec18fd3ec4ef" containerName="istio-proxy" Jul 02 23:37:26.809055 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.808972 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a38864b7-f740-43e5-91bb-ec18fd3ec4ef" containerName="istio-proxy" Jul 02 23:37:26.809055 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.808981 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a38864b7-f740-43e5-91bb-ec18fd3ec4ef" containerName="istio-proxy" Jul 02 23:37:26.813106 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.813081 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.815066 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.815048 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-dbb152-openshift-default-dockercfg-lk4f2\"" Jul 02 23:37:26.824527 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.824502 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4"] Jul 02 23:37:26.879195 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.879143 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-token\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.879195 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.879199 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-podinfo\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.879470 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.879297 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-credential-socket\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.879470 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.879352 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-data\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.879470 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.879394 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istiod-ca-cert\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.879470 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.879445 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-envoy\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.879470 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.879468 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99l6x\" (UniqueName: \"kubernetes.io/projected/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-kube-api-access-99l6x\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.879649 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.879512 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-workload-certs\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.879649 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.879549 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-workload-socket\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.980964 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.980917 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-token\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.981143 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.980979 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-podinfo\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.981143 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.981012 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-credential-socket\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.981143 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.981054 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-data\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.981143 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.981100 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istiod-ca-cert\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.981143 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.981136 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-envoy\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.981410 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.981162 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99l6x\" (UniqueName: \"kubernetes.io/projected/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-kube-api-access-99l6x\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.981410 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.981197 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-workload-certs\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.981410 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.981235 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-workload-socket\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.981575 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.981470 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-credential-socket\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.981575 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.981506 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-data\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.981684 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.981603 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-workload-socket\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.981742 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.981686 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-workload-certs\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.982117 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.982094 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istiod-ca-cert\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.983545 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.983516 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-envoy\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.984402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.984054 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-podinfo\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.990259 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.990238 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99l6x\" (UniqueName: \"kubernetes.io/projected/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-kube-api-access-99l6x\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:26.990592 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:26.990573 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-token\") pod \"e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:27.124043 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:27.123947 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:27.262914 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:27.262874 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4"] Jul 02 23:37:27.265286 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:37:27.265249 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b7c30b8_e420_466f_b716_aa6f5c5a23b9.slice/crio-da9762b27f0cd60eef1c9ea7bc452ec05f1c79686b96e9b33208384f7a599db3 WatchSource:0}: Error finding container da9762b27f0cd60eef1c9ea7bc452ec05f1c79686b96e9b33208384f7a599db3: Status 404 returned error can't find the container with id da9762b27f0cd60eef1c9ea7bc452ec05f1c79686b96e9b33208384f7a599db3 Jul 02 23:37:27.267541 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:27.267510 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:37:27.267622 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:27.267573 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:37:27.267622 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:27.267602 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:37:28.094296 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:28.094255 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" event={"ID":"8b7c30b8-e420-466f-b716-aa6f5c5a23b9","Type":"ContainerStarted","Data":"9e3a4bc2e27140da8b0ba8e2d021095048f80faa2d70e3df504f4cd20e062a48"} Jul 02 23:37:28.094765 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:28.094334 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" event={"ID":"8b7c30b8-e420-466f-b716-aa6f5c5a23b9","Type":"ContainerStarted","Data":"da9762b27f0cd60eef1c9ea7bc452ec05f1c79686b96e9b33208384f7a599db3"} Jul 02 23:37:28.112443 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:28.112383 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" podStartSLOduration=2.112369292 podStartE2EDuration="2.112369292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:37:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:37:28.111131635 +0000 UTC m=+1812.133949751" watchObservedRunningTime="2026-07-02 23:37:28.112369292 +0000 UTC m=+1812.135187404" Jul 02 23:37:28.124956 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:28.124932 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:28.129930 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:28.129877 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:29.099138 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:29.099108 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:29.100124 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:29.100102 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:35.184200 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.184159 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w"] Jul 02 23:37:35.188552 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.188524 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.190775 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.190735 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-2998a4-openshift-default-dockercfg-hntdc\"" Jul 02 23:37:35.198146 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.198122 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w"] Jul 02 23:37:35.254008 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.253962 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-workload-socket\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.254214 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.254015 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvwwk\" (UniqueName: \"kubernetes.io/projected/83493128-4dc2-4a78-ac40-b94646e9585c-kube-api-access-dvwwk\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.254214 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.254107 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-istio-data\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.254214 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.254193 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-istio-envoy\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.254342 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.254231 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83493128-4dc2-4a78-ac40-b94646e9585c-istio-podinfo\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.254342 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.254255 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83493128-4dc2-4a78-ac40-b94646e9585c-istio-token\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.254420 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.254336 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-workload-certs\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.254420 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.254378 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83493128-4dc2-4a78-ac40-b94646e9585c-istiod-ca-cert\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.254420 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.254405 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-credential-socket\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.355087 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.355047 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-credential-socket\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.355269 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.355101 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-workload-socket\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.355269 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.355135 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvwwk\" (UniqueName: \"kubernetes.io/projected/83493128-4dc2-4a78-ac40-b94646e9585c-kube-api-access-dvwwk\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.355269 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.355172 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-istio-data\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.355375 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.355323 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-istio-envoy\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.355425 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.355382 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83493128-4dc2-4a78-ac40-b94646e9585c-istio-podinfo\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.355425 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.355405 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83493128-4dc2-4a78-ac40-b94646e9585c-istio-token\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.355515 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.355456 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-workload-certs\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.355515 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.355478 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83493128-4dc2-4a78-ac40-b94646e9585c-istiod-ca-cert\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.355515 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.355484 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-credential-socket\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.355671 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.355536 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-istio-data\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.355671 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.355565 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-workload-socket\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.355787 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.355751 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-workload-certs\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.356239 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.356168 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83493128-4dc2-4a78-ac40-b94646e9585c-istiod-ca-cert\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.357700 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.357674 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-istio-envoy\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.357920 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.357881 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83493128-4dc2-4a78-ac40-b94646e9585c-istio-podinfo\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.362736 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.362714 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvwwk\" (UniqueName: \"kubernetes.io/projected/83493128-4dc2-4a78-ac40-b94646e9585c-kube-api-access-dvwwk\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.363291 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.363269 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83493128-4dc2-4a78-ac40-b94646e9585c-istio-token\") pod \"e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.501438 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.501354 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:35.639051 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.639026 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w"] Jul 02 23:37:35.640999 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:37:35.640967 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83493128_4dc2_4a78_ac40_b94646e9585c.slice/crio-02ec1ac6464d42341bd180202ff07284633d3d0fcb87c32e20cf0589476e987b WatchSource:0}: Error finding container 02ec1ac6464d42341bd180202ff07284633d3d0fcb87c32e20cf0589476e987b: Status 404 returned error can't find the container with id 02ec1ac6464d42341bd180202ff07284633d3d0fcb87c32e20cf0589476e987b Jul 02 23:37:35.643097 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.643067 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:37:35.643177 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.643133 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:37:35.643177 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:35.643159 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:37:36.134812 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:36.134773 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" event={"ID":"83493128-4dc2-4a78-ac40-b94646e9585c","Type":"ContainerStarted","Data":"5e580725f7f11ee3bb138ca58cde12ec52675c2fa39fadd2edfeebc496d10e8e"} Jul 02 23:37:36.135035 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:36.134837 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" event={"ID":"83493128-4dc2-4a78-ac40-b94646e9585c","Type":"ContainerStarted","Data":"02ec1ac6464d42341bd180202ff07284633d3d0fcb87c32e20cf0589476e987b"} Jul 02 23:37:36.153986 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:36.153928 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" podStartSLOduration=1.153878486 podStartE2EDuration="1.153878486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:37:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:37:36.152864725 +0000 UTC m=+1820.175682853" watchObservedRunningTime="2026-07-02 23:37:36.153878486 +0000 UTC m=+1820.176696600" Jul 02 23:37:36.502481 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:36.502372 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:36.507297 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:36.507266 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:37.139122 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:37.139087 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:37.140077 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:37.140057 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:45.141137 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:45.141084 2583 patch_prober.go:28] interesting pod/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 02 23:37:45.141648 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:45.141171 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" podUID="8b7c30b8-e420-466f-b716-aa6f5c5a23b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 02 23:37:46.191337 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:46.190114 2583 generic.go:356] "Generic (PLEG): container finished" podID="8b7c30b8-e420-466f-b716-aa6f5c5a23b9" containerID="9e3a4bc2e27140da8b0ba8e2d021095048f80faa2d70e3df504f4cd20e062a48" exitCode=0 Jul 02 23:37:46.191337 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:46.190214 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" event={"ID":"8b7c30b8-e420-466f-b716-aa6f5c5a23b9","Type":"ContainerDied","Data":"9e3a4bc2e27140da8b0ba8e2d021095048f80faa2d70e3df504f4cd20e062a48"} Jul 02 23:37:46.191337 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:46.190691 2583 scope.go:117] "RemoveContainer" containerID="9e3a4bc2e27140da8b0ba8e2d021095048f80faa2d70e3df504f4cd20e062a48" Jul 02 23:37:46.194376 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:46.192953 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:37:46.194376 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:46.193043 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:37:46.194376 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:46.193093 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:37:47.200595 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:47.200544 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" event={"ID":"8b7c30b8-e420-466f-b716-aa6f5c5a23b9","Type":"ContainerStarted","Data":"d5ea91b860c9e3d65be56a4a5af94150fa878692a7e4cf2f180cf028a211e23f"} Jul 02 23:37:48.124668 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:48.124621 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:48.124860 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:48.124763 2583 patch_prober.go:28] interesting pod/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 02 23:37:48.124860 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:48.124808 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" podUID="8b7c30b8-e420-466f-b716-aa6f5c5a23b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 02 23:37:49.125238 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:49.125205 2583 patch_prober.go:28] interesting pod/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 02 23:37:49.125696 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:49.125261 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" podUID="8b7c30b8-e420-466f-b716-aa6f5c5a23b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 02 23:37:50.124394 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:50.124344 2583 patch_prober.go:28] interesting pod/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 02 23:37:50.124580 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:50.124410 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" podUID="8b7c30b8-e420-466f-b716-aa6f5c5a23b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 02 23:37:51.124521 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:51.124478 2583 patch_prober.go:28] interesting pod/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 02 23:37:51.124931 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:51.124543 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" podUID="8b7c30b8-e420-466f-b716-aa6f5c5a23b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 02 23:37:52.127763 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:52.127729 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:52.129229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:52.128003 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:52.129229 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:52.128603 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:37:52.142429 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:52.142398 2583 patch_prober.go:28] interesting pod/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": read tcp 10.133.0.2:52466->10.133.0.51:15021: read: connection reset by peer" start-of-body= Jul 02 23:37:52.142562 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:52.142466 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" podUID="83493128-4dc2-4a78-ac40-b94646e9585c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": read tcp 10.133.0.2:52466->10.133.0.51:15021: read: connection reset by peer" Jul 02 23:37:52.229164 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:52.229135 2583 generic.go:356] "Generic (PLEG): container finished" podID="83493128-4dc2-4a78-ac40-b94646e9585c" containerID="5e580725f7f11ee3bb138ca58cde12ec52675c2fa39fadd2edfeebc496d10e8e" exitCode=0 Jul 02 23:37:52.229326 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:52.229208 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" event={"ID":"83493128-4dc2-4a78-ac40-b94646e9585c","Type":"ContainerDied","Data":"5e580725f7f11ee3bb138ca58cde12ec52675c2fa39fadd2edfeebc496d10e8e"} Jul 02 23:37:52.229664 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:52.229647 2583 scope.go:117] "RemoveContainer" containerID="5e580725f7f11ee3bb138ca58cde12ec52675c2fa39fadd2edfeebc496d10e8e" Jul 02 23:37:52.230582 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:52.230556 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:37:52.230675 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:52.230634 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:37:52.230748 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:52.230688 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:37:53.236027 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:53.235986 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" event={"ID":"83493128-4dc2-4a78-ac40-b94646e9585c","Type":"ContainerStarted","Data":"c24752eff5806f6f232725275ad90057244dba6cfdd60f333c3396105f9f0765"} Jul 02 23:37:53.502367 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:53.502263 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:53.502586 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:53.502558 2583 patch_prober.go:28] interesting pod/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 02 23:37:53.502650 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:53.502624 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" podUID="83493128-4dc2-4a78-ac40-b94646e9585c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 02 23:37:54.502052 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:54.502013 2583 patch_prober.go:28] interesting pod/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 02 23:37:54.502465 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:54.502073 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" podUID="83493128-4dc2-4a78-ac40-b94646e9585c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 02 23:37:55.502781 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:55.502741 2583 patch_prober.go:28] interesting pod/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 02 23:37:55.503225 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:55.502815 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" podUID="83493128-4dc2-4a78-ac40-b94646e9585c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 02 23:37:56.501782 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:56.501746 2583 patch_prober.go:28] interesting pod/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 02 23:37:56.502008 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:56.501802 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" podUID="83493128-4dc2-4a78-ac40-b94646e9585c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 02 23:37:57.505820 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:57.505782 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:57.506299 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:57.506074 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:37:57.506745 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:37:57.506728 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:38:36.833969 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:36.833836 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8"] Jul 02 23:38:36.836689 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:36.836667 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:36.839776 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:36.839752 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2998a4\"/\"openshift-service-ca.crt\"" Jul 02 23:38:36.839929 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:36.839778 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2998a4\"/\"default-dockercfg-d986w\"" Jul 02 23:38:36.839929 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:36.839752 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2998a4\"/\"kube-root-ca.crt\"" Jul 02 23:38:36.839929 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:36.839752 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2998a4\"/\"test-model-2998a4-kserve-self-signed-certs\"" Jul 02 23:38:36.848242 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:36.848220 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8"] Jul 02 23:38:36.948222 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:36.948168 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-dshm\") pod \"test-model-2998a4-kserve-6dc45f4f9f-tccw8\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:36.948407 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:36.948283 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79e77b65-9230-47da-b377-fd0ac6744616-tls-certs\") pod \"test-model-2998a4-kserve-6dc45f4f9f-tccw8\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:36.948407 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:36.948346 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-tmp-dir\") pod \"test-model-2998a4-kserve-6dc45f4f9f-tccw8\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:36.948407 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:36.948372 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m76l5\" (UniqueName: \"kubernetes.io/projected/79e77b65-9230-47da-b377-fd0ac6744616-kube-api-access-m76l5\") pod \"test-model-2998a4-kserve-6dc45f4f9f-tccw8\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:36.948531 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:36.948418 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-home\") pod \"test-model-2998a4-kserve-6dc45f4f9f-tccw8\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:36.948531 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:36.948442 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-model-cache\") pod \"test-model-2998a4-kserve-6dc45f4f9f-tccw8\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:37.048994 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:37.048947 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-dshm\") pod \"test-model-2998a4-kserve-6dc45f4f9f-tccw8\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:37.049189 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:37.049014 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79e77b65-9230-47da-b377-fd0ac6744616-tls-certs\") pod \"test-model-2998a4-kserve-6dc45f4f9f-tccw8\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:37.049189 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:37.049064 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-tmp-dir\") pod \"test-model-2998a4-kserve-6dc45f4f9f-tccw8\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:37.049189 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:37.049088 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m76l5\" (UniqueName: \"kubernetes.io/projected/79e77b65-9230-47da-b377-fd0ac6744616-kube-api-access-m76l5\") pod \"test-model-2998a4-kserve-6dc45f4f9f-tccw8\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:37.049359 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:37.049223 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-home\") pod \"test-model-2998a4-kserve-6dc45f4f9f-tccw8\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:37.049359 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:37.049271 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-model-cache\") pod \"test-model-2998a4-kserve-6dc45f4f9f-tccw8\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:37.049467 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:37.049441 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-tmp-dir\") pod \"test-model-2998a4-kserve-6dc45f4f9f-tccw8\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:37.049604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:37.049577 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-home\") pod \"test-model-2998a4-kserve-6dc45f4f9f-tccw8\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:37.049673 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:37.049649 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-model-cache\") pod \"test-model-2998a4-kserve-6dc45f4f9f-tccw8\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:37.051460 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:37.051438 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-dshm\") pod \"test-model-2998a4-kserve-6dc45f4f9f-tccw8\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:37.051703 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:37.051684 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79e77b65-9230-47da-b377-fd0ac6744616-tls-certs\") pod \"test-model-2998a4-kserve-6dc45f4f9f-tccw8\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:37.056271 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:37.056246 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m76l5\" (UniqueName: \"kubernetes.io/projected/79e77b65-9230-47da-b377-fd0ac6744616-kube-api-access-m76l5\") pod \"test-model-2998a4-kserve-6dc45f4f9f-tccw8\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:37.148349 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:37.148262 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:37.294061 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:37.294034 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8"] Jul 02 23:38:37.296175 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:38:37.296145 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79e77b65_9230_47da_b377_fd0ac6744616.slice/crio-849975e094dc67e096cbdcd705a5b8ddd5c722d6075ac9c589f036c5a0a9f390 WatchSource:0}: Error finding container 849975e094dc67e096cbdcd705a5b8ddd5c722d6075ac9c589f036c5a0a9f390: Status 404 returned error can't find the container with id 849975e094dc67e096cbdcd705a5b8ddd5c722d6075ac9c589f036c5a0a9f390 Jul 02 23:38:37.452939 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:37.452829 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" event={"ID":"79e77b65-9230-47da-b377-fd0ac6744616","Type":"ContainerStarted","Data":"42c3ce832c89b7a36ae5b11a1721284345b130ed12a80f86455e65117923f13e"} Jul 02 23:38:37.452939 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:37.452883 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" event={"ID":"79e77b65-9230-47da-b377-fd0ac6744616","Type":"ContainerStarted","Data":"849975e094dc67e096cbdcd705a5b8ddd5c722d6075ac9c589f036c5a0a9f390"} Jul 02 23:38:37.468240 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:37.468190 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" podStartSLOduration=1.468172332 podStartE2EDuration="1.468172332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:38:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:38:37.467094481 +0000 UTC m=+1881.489912607" watchObservedRunningTime="2026-07-02 23:38:37.468172332 +0000 UTC m=+1881.490990444" Jul 02 23:38:47.148877 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:47.148816 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:47.149411 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:47.148926 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:47.161853 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:47.161825 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:38:47.515977 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:38:47.515859 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:39:02.348705 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:02.348664 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8"] Jul 02 23:39:17.370149 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.370093 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" podUID="79e77b65-9230-47da-b377-fd0ac6744616" containerName="main" containerID="cri-o://42c3ce832c89b7a36ae5b11a1721284345b130ed12a80f86455e65117923f13e" gracePeriod=45 Jul 02 23:39:17.505103 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.505053 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" podUID="79e77b65-9230-47da-b377-fd0ac6744616" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": dial tcp 10.133.0.52:8000: connect: connection refused" Jul 02 23:39:17.624327 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.624267 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:39:17.648442 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.648408 2583 generic.go:356] "Generic (PLEG): container finished" podID="79e77b65-9230-47da-b377-fd0ac6744616" containerID="42c3ce832c89b7a36ae5b11a1721284345b130ed12a80f86455e65117923f13e" exitCode=0 Jul 02 23:39:17.648601 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.648454 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" Jul 02 23:39:17.648601 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.648487 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" event={"ID":"79e77b65-9230-47da-b377-fd0ac6744616","Type":"ContainerDied","Data":"42c3ce832c89b7a36ae5b11a1721284345b130ed12a80f86455e65117923f13e"} Jul 02 23:39:17.648601 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.648526 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8" event={"ID":"79e77b65-9230-47da-b377-fd0ac6744616","Type":"ContainerDied","Data":"849975e094dc67e096cbdcd705a5b8ddd5c722d6075ac9c589f036c5a0a9f390"} Jul 02 23:39:17.648601 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.648546 2583 scope.go:117] "RemoveContainer" containerID="42c3ce832c89b7a36ae5b11a1721284345b130ed12a80f86455e65117923f13e" Jul 02 23:39:17.657143 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.657121 2583 scope.go:117] "RemoveContainer" containerID="42c3ce832c89b7a36ae5b11a1721284345b130ed12a80f86455e65117923f13e" Jul 02 23:39:17.657416 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:39:17.657392 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42c3ce832c89b7a36ae5b11a1721284345b130ed12a80f86455e65117923f13e\": container with ID starting with 42c3ce832c89b7a36ae5b11a1721284345b130ed12a80f86455e65117923f13e not found: ID does not exist" containerID="42c3ce832c89b7a36ae5b11a1721284345b130ed12a80f86455e65117923f13e" Jul 02 23:39:17.657482 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.657431 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42c3ce832c89b7a36ae5b11a1721284345b130ed12a80f86455e65117923f13e"} err="failed to get container status \"42c3ce832c89b7a36ae5b11a1721284345b130ed12a80f86455e65117923f13e\": rpc error: code = NotFound desc = could not find container \"42c3ce832c89b7a36ae5b11a1721284345b130ed12a80f86455e65117923f13e\": container with ID starting with 42c3ce832c89b7a36ae5b11a1721284345b130ed12a80f86455e65117923f13e not found: ID does not exist" Jul 02 23:39:17.727795 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.727763 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-home\") pod \"79e77b65-9230-47da-b377-fd0ac6744616\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " Jul 02 23:39:17.727795 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.727807 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-dshm\") pod \"79e77b65-9230-47da-b377-fd0ac6744616\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " Jul 02 23:39:17.728076 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.727836 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m76l5\" (UniqueName: \"kubernetes.io/projected/79e77b65-9230-47da-b377-fd0ac6744616-kube-api-access-m76l5\") pod \"79e77b65-9230-47da-b377-fd0ac6744616\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " Jul 02 23:39:17.728076 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.727963 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79e77b65-9230-47da-b377-fd0ac6744616-tls-certs\") pod \"79e77b65-9230-47da-b377-fd0ac6744616\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " Jul 02 23:39:17.728076 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.727989 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-tmp-dir\") pod \"79e77b65-9230-47da-b377-fd0ac6744616\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " Jul 02 23:39:17.728076 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.728017 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-model-cache\") pod \"79e77b65-9230-47da-b377-fd0ac6744616\" (UID: \"79e77b65-9230-47da-b377-fd0ac6744616\") " Jul 02 23:39:17.728297 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.728132 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-home" (OuterVolumeSpecName: "home") pod "79e77b65-9230-47da-b377-fd0ac6744616" (UID: "79e77b65-9230-47da-b377-fd0ac6744616"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:17.728297 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.728277 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "79e77b65-9230-47da-b377-fd0ac6744616" (UID: "79e77b65-9230-47da-b377-fd0ac6744616"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:17.728396 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.728314 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-model-cache" (OuterVolumeSpecName: "model-cache") pod "79e77b65-9230-47da-b377-fd0ac6744616" (UID: "79e77b65-9230-47da-b377-fd0ac6744616"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:17.728439 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.728406 2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-home\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:17.728439 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.728425 2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-tmp-dir\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:17.728510 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.728440 2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-model-cache\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:17.730402 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.730372 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-dshm" (OuterVolumeSpecName: "dshm") pod "79e77b65-9230-47da-b377-fd0ac6744616" (UID: "79e77b65-9230-47da-b377-fd0ac6744616"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:17.730614 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.730591 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79e77b65-9230-47da-b377-fd0ac6744616-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "79e77b65-9230-47da-b377-fd0ac6744616" (UID: "79e77b65-9230-47da-b377-fd0ac6744616"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:39:17.730662 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.730627 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79e77b65-9230-47da-b377-fd0ac6744616-kube-api-access-m76l5" (OuterVolumeSpecName: "kube-api-access-m76l5") pod "79e77b65-9230-47da-b377-fd0ac6744616" (UID: "79e77b65-9230-47da-b377-fd0ac6744616"). InnerVolumeSpecName "kube-api-access-m76l5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:39:17.829876 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.829840 2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79e77b65-9230-47da-b377-fd0ac6744616-dshm\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:17.829876 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.829870 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m76l5\" (UniqueName: \"kubernetes.io/projected/79e77b65-9230-47da-b377-fd0ac6744616-kube-api-access-m76l5\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:17.829876 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.829880 2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79e77b65-9230-47da-b377-fd0ac6744616-tls-certs\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:17.973721 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.973665 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8"] Jul 02 23:39:17.975219 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:17.975188 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-2998a4/test-model-2998a4-kserve-6dc45f4f9f-tccw8"] Jul 02 23:39:18.532910 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:18.532867 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79e77b65-9230-47da-b377-fd0ac6744616" path="/var/lib/kubelet/pods/79e77b65-9230-47da-b377-fd0ac6744616/volumes" Jul 02 23:39:44.845458 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:44.845419 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4"] Jul 02 23:39:44.846049 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:44.845612 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" podUID="8b7c30b8-e420-466f-b716-aa6f5c5a23b9" containerName="istio-proxy" containerID="cri-o://d5ea91b860c9e3d65be56a4a5af94150fa878692a7e4cf2f180cf028a211e23f" gracePeriod=30 Jul 02 23:39:50.109158 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.109084 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:39:50.223367 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.223323 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-workload-certs\") pod \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " Jul 02 23:39:50.223367 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.223371 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-token\") pod \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " Jul 02 23:39:50.223607 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.223389 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-envoy\") pod \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " Jul 02 23:39:50.223607 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.223473 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-data\") pod \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " Jul 02 23:39:50.223607 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.223525 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99l6x\" (UniqueName: \"kubernetes.io/projected/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-kube-api-access-99l6x\") pod \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " Jul 02 23:39:50.223607 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.223559 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-podinfo\") pod \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " Jul 02 23:39:50.223607 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.223589 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istiod-ca-cert\") pod \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " Jul 02 23:39:50.223920 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.223612 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-workload-socket\") pod \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " Jul 02 23:39:50.223920 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.223642 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8b7c30b8-e420-466f-b716-aa6f5c5a23b9" (UID: "8b7c30b8-e420-466f-b716-aa6f5c5a23b9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:50.223920 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.223836 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-credential-socket\") pod \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\" (UID: \"8b7c30b8-e420-466f-b716-aa6f5c5a23b9\") " Jul 02 23:39:50.224090 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.223968 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8b7c30b8-e420-466f-b716-aa6f5c5a23b9" (UID: "8b7c30b8-e420-466f-b716-aa6f5c5a23b9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:50.224211 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.224161 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8b7c30b8-e420-466f-b716-aa6f5c5a23b9" (UID: "8b7c30b8-e420-466f-b716-aa6f5c5a23b9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:50.224517 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.224286 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-workload-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:50.224517 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.224303 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-credential-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:50.224517 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.224342 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-workload-certs\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:50.224517 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.224431 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-data" (OuterVolumeSpecName: "istio-data") pod "8b7c30b8-e420-466f-b716-aa6f5c5a23b9" (UID: "8b7c30b8-e420-466f-b716-aa6f5c5a23b9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:50.224517 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.224480 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8b7c30b8-e420-466f-b716-aa6f5c5a23b9" (UID: "8b7c30b8-e420-466f-b716-aa6f5c5a23b9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:39:50.225802 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.225769 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-token" (OuterVolumeSpecName: "istio-token") pod "8b7c30b8-e420-466f-b716-aa6f5c5a23b9" (UID: "8b7c30b8-e420-466f-b716-aa6f5c5a23b9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:39:50.225949 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.225922 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8b7c30b8-e420-466f-b716-aa6f5c5a23b9" (UID: "8b7c30b8-e420-466f-b716-aa6f5c5a23b9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:39:50.226321 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.226305 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8b7c30b8-e420-466f-b716-aa6f5c5a23b9" (UID: "8b7c30b8-e420-466f-b716-aa6f5c5a23b9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:50.226383 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.226360 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-kube-api-access-99l6x" (OuterVolumeSpecName: "kube-api-access-99l6x") pod "8b7c30b8-e420-466f-b716-aa6f5c5a23b9" (UID: "8b7c30b8-e420-466f-b716-aa6f5c5a23b9"). InnerVolumeSpecName "kube-api-access-99l6x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:39:50.325255 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.325209 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-data\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:50.325255 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.325246 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99l6x\" (UniqueName: \"kubernetes.io/projected/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-kube-api-access-99l6x\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:50.325255 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.325261 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-podinfo\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:50.325506 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.325273 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istiod-ca-cert\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:50.325506 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.325285 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-token\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:50.325506 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.325295 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b7c30b8-e420-466f-b716-aa6f5c5a23b9-istio-envoy\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:50.810552 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.810516 2583 generic.go:356] "Generic (PLEG): container finished" podID="8b7c30b8-e420-466f-b716-aa6f5c5a23b9" containerID="d5ea91b860c9e3d65be56a4a5af94150fa878692a7e4cf2f180cf028a211e23f" exitCode=0 Jul 02 23:39:50.810747 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.810561 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" Jul 02 23:39:50.810747 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.810612 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" event={"ID":"8b7c30b8-e420-466f-b716-aa6f5c5a23b9","Type":"ContainerDied","Data":"d5ea91b860c9e3d65be56a4a5af94150fa878692a7e4cf2f180cf028a211e23f"} Jul 02 23:39:50.810747 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.810664 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4" event={"ID":"8b7c30b8-e420-466f-b716-aa6f5c5a23b9","Type":"ContainerDied","Data":"da9762b27f0cd60eef1c9ea7bc452ec05f1c79686b96e9b33208384f7a599db3"} Jul 02 23:39:50.810747 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.810687 2583 scope.go:117] "RemoveContainer" containerID="d5ea91b860c9e3d65be56a4a5af94150fa878692a7e4cf2f180cf028a211e23f" Jul 02 23:39:50.819792 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.819774 2583 scope.go:117] "RemoveContainer" containerID="9e3a4bc2e27140da8b0ba8e2d021095048f80faa2d70e3df504f4cd20e062a48" Jul 02 23:39:50.827848 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.827818 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4"] Jul 02 23:39:50.828485 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.828464 2583 scope.go:117] "RemoveContainer" containerID="d5ea91b860c9e3d65be56a4a5af94150fa878692a7e4cf2f180cf028a211e23f" Jul 02 23:39:50.828771 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:39:50.828751 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5ea91b860c9e3d65be56a4a5af94150fa878692a7e4cf2f180cf028a211e23f\": container with ID starting with d5ea91b860c9e3d65be56a4a5af94150fa878692a7e4cf2f180cf028a211e23f not found: ID does not exist" containerID="d5ea91b860c9e3d65be56a4a5af94150fa878692a7e4cf2f180cf028a211e23f" Jul 02 23:39:50.828851 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.828779 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5ea91b860c9e3d65be56a4a5af94150fa878692a7e4cf2f180cf028a211e23f"} err="failed to get container status \"d5ea91b860c9e3d65be56a4a5af94150fa878692a7e4cf2f180cf028a211e23f\": rpc error: code = NotFound desc = could not find container \"d5ea91b860c9e3d65be56a4a5af94150fa878692a7e4cf2f180cf028a211e23f\": container with ID starting with d5ea91b860c9e3d65be56a4a5af94150fa878692a7e4cf2f180cf028a211e23f not found: ID does not exist" Jul 02 23:39:50.828851 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.828796 2583 scope.go:117] "RemoveContainer" containerID="9e3a4bc2e27140da8b0ba8e2d021095048f80faa2d70e3df504f4cd20e062a48" Jul 02 23:39:50.829084 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:39:50.829065 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e3a4bc2e27140da8b0ba8e2d021095048f80faa2d70e3df504f4cd20e062a48\": container with ID starting with 9e3a4bc2e27140da8b0ba8e2d021095048f80faa2d70e3df504f4cd20e062a48 not found: ID does not exist" containerID="9e3a4bc2e27140da8b0ba8e2d021095048f80faa2d70e3df504f4cd20e062a48" Jul 02 23:39:50.829146 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.829088 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e3a4bc2e27140da8b0ba8e2d021095048f80faa2d70e3df504f4cd20e062a48"} err="failed to get container status \"9e3a4bc2e27140da8b0ba8e2d021095048f80faa2d70e3df504f4cd20e062a48\": rpc error: code = NotFound desc = could not find container \"9e3a4bc2e27140da8b0ba8e2d021095048f80faa2d70e3df504f4cd20e062a48\": container with ID starting with 9e3a4bc2e27140da8b0ba8e2d021095048f80faa2d70e3df504f4cd20e062a48 not found: ID does not exist" Jul 02 23:39:50.829851 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:50.829829 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-dbb152-openshift-default-54cbbd8f8f-75jw4"] Jul 02 23:39:51.658413 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:51.658381 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w"] Jul 02 23:39:51.658879 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:51.658622 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" podUID="83493128-4dc2-4a78-ac40-b94646e9585c" containerName="istio-proxy" containerID="cri-o://c24752eff5806f6f232725275ad90057244dba6cfdd60f333c3396105f9f0765" gracePeriod=30 Jul 02 23:39:52.511990 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.511920 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf"] Jul 02 23:39:52.513085 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.513056 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79e77b65-9230-47da-b377-fd0ac6744616" containerName="main" Jul 02 23:39:52.513085 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.513085 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="79e77b65-9230-47da-b377-fd0ac6744616" containerName="main" Jul 02 23:39:52.513275 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.513137 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b7c30b8-e420-466f-b716-aa6f5c5a23b9" containerName="istio-proxy" Jul 02 23:39:52.513275 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.513152 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b7c30b8-e420-466f-b716-aa6f5c5a23b9" containerName="istio-proxy" Jul 02 23:39:52.513363 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.513313 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="79e77b65-9230-47da-b377-fd0ac6744616" containerName="main" Jul 02 23:39:52.513363 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.513330 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b7c30b8-e420-466f-b716-aa6f5c5a23b9" containerName="istio-proxy" Jul 02 23:39:52.513363 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.513346 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b7c30b8-e420-466f-b716-aa6f5c5a23b9" containerName="istio-proxy" Jul 02 23:39:52.519789 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.519762 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.522120 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.522093 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-979f9f-openshift-default-dockercfg-rjs4g\"" Jul 02 23:39:52.536760 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.536732 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b7c30b8-e420-466f-b716-aa6f5c5a23b9" path="/var/lib/kubelet/pods/8b7c30b8-e420-466f-b716-aa6f5c5a23b9/volumes" Jul 02 23:39:52.537073 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.537058 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf"] Jul 02 23:39:52.646032 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.645994 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-envoy\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.646226 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.646049 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99fbf8e6-19fa-4d9b-a816-5b1425172932-istiod-ca-cert\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.646226 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.646095 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2m27\" (UniqueName: \"kubernetes.io/projected/99fbf8e6-19fa-4d9b-a816-5b1425172932-kube-api-access-h2m27\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.646226 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.646131 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-podinfo\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.646226 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.646153 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-credential-socket\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.646226 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.646200 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-workload-socket\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.646226 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.646215 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-data\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.646517 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.646235 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-workload-certs\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.646517 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.646308 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-token\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.747438 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.747394 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-envoy\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.747988 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.747476 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99fbf8e6-19fa-4d9b-a816-5b1425172932-istiod-ca-cert\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.747988 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.747521 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2m27\" (UniqueName: \"kubernetes.io/projected/99fbf8e6-19fa-4d9b-a816-5b1425172932-kube-api-access-h2m27\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.747988 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.747577 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-podinfo\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.747988 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.747617 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-credential-socket\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.747988 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.747756 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-workload-socket\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.747988 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.747804 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-data\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.747988 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.747842 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-workload-certs\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.747988 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.747874 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-token\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.747988 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.747982 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-credential-socket\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.748394 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.748218 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-workload-socket\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.748501 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.748481 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-workload-certs\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.748559 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.748523 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99fbf8e6-19fa-4d9b-a816-5b1425172932-istiod-ca-cert\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.748679 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.748658 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-data\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.750130 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.750107 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-envoy\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.750228 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.750195 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-podinfo\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.755491 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.755465 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-token\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.755659 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.755639 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2m27\" (UniqueName: \"kubernetes.io/projected/99fbf8e6-19fa-4d9b-a816-5b1425172932-kube-api-access-h2m27\") pod \"e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.833924 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.833849 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:52.975184 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.975153 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf"] Jul 02 23:39:52.977936 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:39:52.977870 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99fbf8e6_19fa_4d9b_a816_5b1425172932.slice/crio-2abbdb29951ef46ddda2a95a3e07b765e8369f48bc7ed28d790241b3a8d67212 WatchSource:0}: Error finding container 2abbdb29951ef46ddda2a95a3e07b765e8369f48bc7ed28d790241b3a8d67212: Status 404 returned error can't find the container with id 2abbdb29951ef46ddda2a95a3e07b765e8369f48bc7ed28d790241b3a8d67212 Jul 02 23:39:52.980121 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.980089 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:39:52.980210 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.980164 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:39:52.980210 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:52.980194 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:39:53.828548 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:53.828515 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" event={"ID":"99fbf8e6-19fa-4d9b-a816-5b1425172932","Type":"ContainerStarted","Data":"06883aea9c9822f91769a79e1b89b36999e37079b697c32a911cf8d4393a8eaf"} Jul 02 23:39:53.828960 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:53.828563 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" event={"ID":"99fbf8e6-19fa-4d9b-a816-5b1425172932","Type":"ContainerStarted","Data":"2abbdb29951ef46ddda2a95a3e07b765e8369f48bc7ed28d790241b3a8d67212"} Jul 02 23:39:53.834308 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:53.834291 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:53.848218 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:53.848172 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" podStartSLOduration=1.8481577 podStartE2EDuration="1.8481577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:39:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:39:53.846226987 +0000 UTC m=+1957.869045099" watchObservedRunningTime="2026-07-02 23:39:53.8481577 +0000 UTC m=+1957.870975813" Jul 02 23:39:54.839097 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:54.839071 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:55.838137 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:55.838107 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:55.839061 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:55.839041 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:39:56.846870 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.846837 2583 generic.go:356] "Generic (PLEG): container finished" podID="83493128-4dc2-4a78-ac40-b94646e9585c" containerID="c24752eff5806f6f232725275ad90057244dba6cfdd60f333c3396105f9f0765" exitCode=0 Jul 02 23:39:56.847345 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.846935 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" event={"ID":"83493128-4dc2-4a78-ac40-b94646e9585c","Type":"ContainerDied","Data":"c24752eff5806f6f232725275ad90057244dba6cfdd60f333c3396105f9f0765"} Jul 02 23:39:56.847345 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.847020 2583 scope.go:117] "RemoveContainer" containerID="5e580725f7f11ee3bb138ca58cde12ec52675c2fa39fadd2edfeebc496d10e8e" Jul 02 23:39:56.906697 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.906668 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:39:56.991411 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.991316 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-workload-certs\") pod \"83493128-4dc2-4a78-ac40-b94646e9585c\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " Jul 02 23:39:56.991411 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.991367 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvwwk\" (UniqueName: \"kubernetes.io/projected/83493128-4dc2-4a78-ac40-b94646e9585c-kube-api-access-dvwwk\") pod \"83493128-4dc2-4a78-ac40-b94646e9585c\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " Jul 02 23:39:56.991411 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.991388 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-credential-socket\") pod \"83493128-4dc2-4a78-ac40-b94646e9585c\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " Jul 02 23:39:56.991700 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.991419 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-workload-socket\") pod \"83493128-4dc2-4a78-ac40-b94646e9585c\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " Jul 02 23:39:56.991700 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.991445 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83493128-4dc2-4a78-ac40-b94646e9585c-istio-podinfo\") pod \"83493128-4dc2-4a78-ac40-b94646e9585c\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " Jul 02 23:39:56.991700 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.991509 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83493128-4dc2-4a78-ac40-b94646e9585c-istiod-ca-cert\") pod \"83493128-4dc2-4a78-ac40-b94646e9585c\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " Jul 02 23:39:56.991700 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.991538 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83493128-4dc2-4a78-ac40-b94646e9585c-istio-token\") pod \"83493128-4dc2-4a78-ac40-b94646e9585c\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " Jul 02 23:39:56.991700 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.991575 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-istio-data\") pod \"83493128-4dc2-4a78-ac40-b94646e9585c\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " Jul 02 23:39:56.991700 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.991590 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-istio-envoy\") pod \"83493128-4dc2-4a78-ac40-b94646e9585c\" (UID: \"83493128-4dc2-4a78-ac40-b94646e9585c\") " Jul 02 23:39:56.991700 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.991621 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "83493128-4dc2-4a78-ac40-b94646e9585c" (UID: "83493128-4dc2-4a78-ac40-b94646e9585c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:56.992105 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.991814 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "83493128-4dc2-4a78-ac40-b94646e9585c" (UID: "83493128-4dc2-4a78-ac40-b94646e9585c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:56.992105 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.991966 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-workload-certs\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:56.992105 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.991984 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-workload-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:56.992777 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.992590 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83493128-4dc2-4a78-ac40-b94646e9585c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "83493128-4dc2-4a78-ac40-b94646e9585c" (UID: "83493128-4dc2-4a78-ac40-b94646e9585c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:39:56.992777 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.992653 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "83493128-4dc2-4a78-ac40-b94646e9585c" (UID: "83493128-4dc2-4a78-ac40-b94646e9585c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:56.993015 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.992814 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-istio-data" (OuterVolumeSpecName: "istio-data") pod "83493128-4dc2-4a78-ac40-b94646e9585c" (UID: "83493128-4dc2-4a78-ac40-b94646e9585c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:56.993995 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.993965 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83493128-4dc2-4a78-ac40-b94646e9585c-kube-api-access-dvwwk" (OuterVolumeSpecName: "kube-api-access-dvwwk") pod "83493128-4dc2-4a78-ac40-b94646e9585c" (UID: "83493128-4dc2-4a78-ac40-b94646e9585c"). InnerVolumeSpecName "kube-api-access-dvwwk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:39:56.993995 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.993962 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83493128-4dc2-4a78-ac40-b94646e9585c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "83493128-4dc2-4a78-ac40-b94646e9585c" (UID: "83493128-4dc2-4a78-ac40-b94646e9585c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:39:56.994247 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.994229 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83493128-4dc2-4a78-ac40-b94646e9585c-istio-token" (OuterVolumeSpecName: "istio-token") pod "83493128-4dc2-4a78-ac40-b94646e9585c" (UID: "83493128-4dc2-4a78-ac40-b94646e9585c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:39:56.994625 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:56.994607 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "83493128-4dc2-4a78-ac40-b94646e9585c" (UID: "83493128-4dc2-4a78-ac40-b94646e9585c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:57.093178 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:57.093144 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83493128-4dc2-4a78-ac40-b94646e9585c-istio-podinfo\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:57.093178 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:57.093175 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83493128-4dc2-4a78-ac40-b94646e9585c-istiod-ca-cert\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:57.093178 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:57.093185 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83493128-4dc2-4a78-ac40-b94646e9585c-istio-token\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:57.093415 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:57.093193 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-istio-data\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:57.093415 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:57.093200 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-istio-envoy\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:57.093415 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:57.093208 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvwwk\" (UniqueName: \"kubernetes.io/projected/83493128-4dc2-4a78-ac40-b94646e9585c-kube-api-access-dvwwk\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:57.093415 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:57.093218 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83493128-4dc2-4a78-ac40-b94646e9585c-credential-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:39:57.858908 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:57.858853 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" Jul 02 23:39:57.859389 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:57.858913 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w" event={"ID":"83493128-4dc2-4a78-ac40-b94646e9585c","Type":"ContainerDied","Data":"02ec1ac6464d42341bd180202ff07284633d3d0fcb87c32e20cf0589476e987b"} Jul 02 23:39:57.859389 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:57.859014 2583 scope.go:117] "RemoveContainer" containerID="c24752eff5806f6f232725275ad90057244dba6cfdd60f333c3396105f9f0765" Jul 02 23:39:57.882266 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:57.882239 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w"] Jul 02 23:39:57.886554 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:57.886520 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-2998a4-openshift-default-56cbc98d9c-m9r5w"] Jul 02 23:39:58.532873 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:39:58.532839 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83493128-4dc2-4a78-ac40-b94646e9585c" path="/var/lib/kubelet/pods/83493128-4dc2-4a78-ac40-b94646e9585c/volumes" Jul 02 23:40:00.984879 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:00.984836 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg"] Jul 02 23:40:00.985372 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:00.985243 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83493128-4dc2-4a78-ac40-b94646e9585c" containerName="istio-proxy" Jul 02 23:40:00.985372 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:00.985256 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="83493128-4dc2-4a78-ac40-b94646e9585c" containerName="istio-proxy" Jul 02 23:40:00.985372 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:00.985263 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83493128-4dc2-4a78-ac40-b94646e9585c" containerName="istio-proxy" Jul 02 23:40:00.985372 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:00.985269 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="83493128-4dc2-4a78-ac40-b94646e9585c" containerName="istio-proxy" Jul 02 23:40:00.985372 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:00.985278 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b7c30b8-e420-466f-b716-aa6f5c5a23b9" containerName="istio-proxy" Jul 02 23:40:00.985372 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:00.985283 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b7c30b8-e420-466f-b716-aa6f5c5a23b9" containerName="istio-proxy" Jul 02 23:40:00.985372 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:00.985351 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="83493128-4dc2-4a78-ac40-b94646e9585c" containerName="istio-proxy" Jul 02 23:40:00.985372 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:00.985359 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="83493128-4dc2-4a78-ac40-b94646e9585c" containerName="istio-proxy" Jul 02 23:40:00.988568 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:00.988547 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:00.990837 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:00.990807 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-6c2ab8-openshift-default-dockercfg-t9zjv\"" Jul 02 23:40:00.999037 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:00.999003 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg"] Jul 02 23:40:01.127380 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.127336 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15ef0459-de5d-4547-93c8-679e0009e53a-istio-podinfo\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.127380 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.127386 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-workload-certs\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.127633 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.127429 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15ef0459-de5d-4547-93c8-679e0009e53a-istiod-ca-cert\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.127633 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.127479 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-credential-socket\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.127633 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.127511 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-workload-socket\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.127633 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.127572 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qv8g\" (UniqueName: \"kubernetes.io/projected/15ef0459-de5d-4547-93c8-679e0009e53a-kube-api-access-7qv8g\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.127633 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.127596 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-istio-data\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.127633 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.127612 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-istio-envoy\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.127977 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.127670 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15ef0459-de5d-4547-93c8-679e0009e53a-istio-token\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.229061 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.229018 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qv8g\" (UniqueName: \"kubernetes.io/projected/15ef0459-de5d-4547-93c8-679e0009e53a-kube-api-access-7qv8g\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.229061 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.229066 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-istio-data\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.229323 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.229106 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-istio-envoy\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.229323 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.229129 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15ef0459-de5d-4547-93c8-679e0009e53a-istio-token\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.229323 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.229159 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15ef0459-de5d-4547-93c8-679e0009e53a-istio-podinfo\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.229323 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.229203 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-workload-certs\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.229323 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.229236 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15ef0459-de5d-4547-93c8-679e0009e53a-istiod-ca-cert\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.229323 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.229302 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-credential-socket\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.229629 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.229344 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-workload-socket\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.229629 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.229518 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-istio-data\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.229732 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.229623 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-workload-certs\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.229732 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.229701 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-workload-socket\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.229732 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.229722 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-credential-socket\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.230071 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.229942 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15ef0459-de5d-4547-93c8-679e0009e53a-istiod-ca-cert\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.231582 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.231552 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-istio-envoy\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.231835 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.231817 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15ef0459-de5d-4547-93c8-679e0009e53a-istio-podinfo\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.236832 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.236761 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15ef0459-de5d-4547-93c8-679e0009e53a-istio-token\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.236978 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.236930 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qv8g\" (UniqueName: \"kubernetes.io/projected/15ef0459-de5d-4547-93c8-679e0009e53a-kube-api-access-7qv8g\") pod \"e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.300514 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.300458 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:01.441927 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.441878 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg"] Jul 02 23:40:01.444447 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:40:01.444418 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15ef0459_de5d_4547_93c8_679e0009e53a.slice/crio-1233afe6a775fc00aca8a5b59e58f54e30ed9b43b88ce377a458d264d0945e13 WatchSource:0}: Error finding container 1233afe6a775fc00aca8a5b59e58f54e30ed9b43b88ce377a458d264d0945e13: Status 404 returned error can't find the container with id 1233afe6a775fc00aca8a5b59e58f54e30ed9b43b88ce377a458d264d0945e13 Jul 02 23:40:01.446645 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.446610 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:01.446720 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.446693 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:01.446755 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.446731 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:01.881902 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.881856 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" event={"ID":"15ef0459-de5d-4547-93c8-679e0009e53a","Type":"ContainerStarted","Data":"640d55c53440cc6bd3708c2d957679bec5f86bbf86512e7d51726244ed99456e"} Jul 02 23:40:01.882095 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.881945 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" event={"ID":"15ef0459-de5d-4547-93c8-679e0009e53a","Type":"ContainerStarted","Data":"1233afe6a775fc00aca8a5b59e58f54e30ed9b43b88ce377a458d264d0945e13"} Jul 02 23:40:01.904737 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:01.904679 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" podStartSLOduration=1.904663186 podStartE2EDuration="1.904663186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:40:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:40:01.90305524 +0000 UTC m=+1965.925873353" watchObservedRunningTime="2026-07-02 23:40:01.904663186 +0000 UTC m=+1965.927481299" Jul 02 23:40:02.300953 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:02.300855 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:02.305670 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:02.305642 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:02.886488 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:02.886450 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:02.887458 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:02.887440 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:11.477261 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:11.477218 2583 patch_prober.go:28] interesting pod/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": read tcp 10.133.0.2:34500->10.133.0.53:15021: read: connection reset by peer" start-of-body= Jul 02 23:40:11.477635 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:11.477287 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" podUID="99fbf8e6-19fa-4d9b-a816-5b1425172932" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": read tcp 10.133.0.2:34500->10.133.0.53:15021: read: connection reset by peer" Jul 02 23:40:11.937076 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:11.937043 2583 generic.go:356] "Generic (PLEG): container finished" podID="99fbf8e6-19fa-4d9b-a816-5b1425172932" containerID="06883aea9c9822f91769a79e1b89b36999e37079b697c32a911cf8d4393a8eaf" exitCode=0 Jul 02 23:40:11.937260 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:11.937092 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" event={"ID":"99fbf8e6-19fa-4d9b-a816-5b1425172932","Type":"ContainerDied","Data":"06883aea9c9822f91769a79e1b89b36999e37079b697c32a911cf8d4393a8eaf"} Jul 02 23:40:11.937493 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:11.937475 2583 scope.go:117] "RemoveContainer" containerID="06883aea9c9822f91769a79e1b89b36999e37079b697c32a911cf8d4393a8eaf" Jul 02 23:40:11.938584 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:11.938552 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:11.938680 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:11.938627 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:11.938680 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:11.938660 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:12.945028 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:12.944991 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" event={"ID":"99fbf8e6-19fa-4d9b-a816-5b1425172932","Type":"ContainerStarted","Data":"b4c5d24b3a15bf455895e88d59389ac48c8eeae6f14998def7e49a54e54fd949"} Jul 02 23:40:13.834940 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:13.834873 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:40:13.835132 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:13.835013 2583 patch_prober.go:28] interesting pod/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 02 23:40:13.835132 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:13.835048 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" podUID="99fbf8e6-19fa-4d9b-a816-5b1425172932" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 02 23:40:14.834574 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:14.834529 2583 patch_prober.go:28] interesting pod/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 02 23:40:14.835011 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:14.834591 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" podUID="99fbf8e6-19fa-4d9b-a816-5b1425172932" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 02 23:40:15.835001 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:15.834954 2583 patch_prober.go:28] interesting pod/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 02 23:40:15.835496 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:15.835017 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" podUID="99fbf8e6-19fa-4d9b-a816-5b1425172932" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 02 23:40:16.835008 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:16.834878 2583 patch_prober.go:28] interesting pod/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 02 23:40:16.835221 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:16.835187 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" podUID="99fbf8e6-19fa-4d9b-a816-5b1425172932" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 02 23:40:17.838346 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:17.838297 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:40:17.838846 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:17.838826 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:40:17.839833 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:17.839799 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:40:18.929114 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:18.929075 2583 patch_prober.go:28] interesting pod/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 02 23:40:18.929491 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:18.929152 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" podUID="15ef0459-de5d-4547-93c8-679e0009e53a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 02 23:40:19.990440 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:19.990407 2583 generic.go:356] "Generic (PLEG): container finished" podID="15ef0459-de5d-4547-93c8-679e0009e53a" containerID="640d55c53440cc6bd3708c2d957679bec5f86bbf86512e7d51726244ed99456e" exitCode=0 Jul 02 23:40:19.990877 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:19.990485 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" event={"ID":"15ef0459-de5d-4547-93c8-679e0009e53a","Type":"ContainerDied","Data":"640d55c53440cc6bd3708c2d957679bec5f86bbf86512e7d51726244ed99456e"} Jul 02 23:40:19.990962 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:19.990914 2583 scope.go:117] "RemoveContainer" containerID="640d55c53440cc6bd3708c2d957679bec5f86bbf86512e7d51726244ed99456e" Jul 02 23:40:19.991944 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:19.991912 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:19.992077 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:19.991979 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:19.992077 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:19.992020 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:21.001170 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:21.001130 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" event={"ID":"15ef0459-de5d-4547-93c8-679e0009e53a","Type":"ContainerStarted","Data":"22c08777b7a2ddfcc66e58072a8a4a72f372ed5f4fb16012c17d17b189541d6c"} Jul 02 23:40:21.301602 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:21.301555 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:21.301832 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:21.301810 2583 patch_prober.go:28] interesting pod/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 02 23:40:21.301878 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:21.301855 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" podUID="15ef0459-de5d-4547-93c8-679e0009e53a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 02 23:40:22.300859 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:22.300814 2583 patch_prober.go:28] interesting pod/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 02 23:40:22.301299 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:22.300878 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" podUID="15ef0459-de5d-4547-93c8-679e0009e53a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 02 23:40:22.576069 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:22.575968 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf"] Jul 02 23:40:22.576236 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:22.576216 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" podUID="99fbf8e6-19fa-4d9b-a816-5b1425172932" containerName="istio-proxy" containerID="cri-o://b4c5d24b3a15bf455895e88d59389ac48c8eeae6f14998def7e49a54e54fd949" gracePeriod=30 Jul 02 23:40:23.301097 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:23.301050 2583 patch_prober.go:28] interesting pod/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 02 23:40:23.301523 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:23.301125 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" podUID="15ef0459-de5d-4547-93c8-679e0009e53a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 02 23:40:24.301295 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:24.301254 2583 patch_prober.go:28] interesting pod/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 02 23:40:24.301674 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:24.301325 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" podUID="15ef0459-de5d-4547-93c8-679e0009e53a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 02 23:40:25.304781 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:25.304748 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:25.305196 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:25.305052 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:25.305746 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:25.305731 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:27.843504 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.843471 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:40:27.882374 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.882289 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-token\") pod \"99fbf8e6-19fa-4d9b-a816-5b1425172932\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " Jul 02 23:40:27.882553 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.882398 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2m27\" (UniqueName: \"kubernetes.io/projected/99fbf8e6-19fa-4d9b-a816-5b1425172932-kube-api-access-h2m27\") pod \"99fbf8e6-19fa-4d9b-a816-5b1425172932\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " Jul 02 23:40:27.882553 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.882422 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99fbf8e6-19fa-4d9b-a816-5b1425172932-istiod-ca-cert\") pod \"99fbf8e6-19fa-4d9b-a816-5b1425172932\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " Jul 02 23:40:27.882553 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.882442 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-workload-certs\") pod \"99fbf8e6-19fa-4d9b-a816-5b1425172932\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " Jul 02 23:40:27.882553 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.882488 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-podinfo\") pod \"99fbf8e6-19fa-4d9b-a816-5b1425172932\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " Jul 02 23:40:27.882553 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.882545 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-envoy\") pod \"99fbf8e6-19fa-4d9b-a816-5b1425172932\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " Jul 02 23:40:27.882937 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.882573 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-workload-socket\") pod \"99fbf8e6-19fa-4d9b-a816-5b1425172932\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " Jul 02 23:40:27.882937 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.882603 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-credential-socket\") pod \"99fbf8e6-19fa-4d9b-a816-5b1425172932\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " Jul 02 23:40:27.882937 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.882626 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-data\") pod \"99fbf8e6-19fa-4d9b-a816-5b1425172932\" (UID: \"99fbf8e6-19fa-4d9b-a816-5b1425172932\") " Jul 02 23:40:27.883109 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.882981 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "99fbf8e6-19fa-4d9b-a816-5b1425172932" (UID: "99fbf8e6-19fa-4d9b-a816-5b1425172932"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:40:27.883276 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.883249 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99fbf8e6-19fa-4d9b-a816-5b1425172932-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "99fbf8e6-19fa-4d9b-a816-5b1425172932" (UID: "99fbf8e6-19fa-4d9b-a816-5b1425172932"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:40:27.883589 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.883368 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "99fbf8e6-19fa-4d9b-a816-5b1425172932" (UID: "99fbf8e6-19fa-4d9b-a816-5b1425172932"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:40:27.883589 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.883543 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "99fbf8e6-19fa-4d9b-a816-5b1425172932" (UID: "99fbf8e6-19fa-4d9b-a816-5b1425172932"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:40:27.884541 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.884511 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-data" (OuterVolumeSpecName: "istio-data") pod "99fbf8e6-19fa-4d9b-a816-5b1425172932" (UID: "99fbf8e6-19fa-4d9b-a816-5b1425172932"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:40:27.885718 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.885693 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "99fbf8e6-19fa-4d9b-a816-5b1425172932" (UID: "99fbf8e6-19fa-4d9b-a816-5b1425172932"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:40:27.885718 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.885693 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99fbf8e6-19fa-4d9b-a816-5b1425172932-kube-api-access-h2m27" (OuterVolumeSpecName: "kube-api-access-h2m27") pod "99fbf8e6-19fa-4d9b-a816-5b1425172932" (UID: "99fbf8e6-19fa-4d9b-a816-5b1425172932"). InnerVolumeSpecName "kube-api-access-h2m27". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:40:27.885857 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.885709 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-token" (OuterVolumeSpecName: "istio-token") pod "99fbf8e6-19fa-4d9b-a816-5b1425172932" (UID: "99fbf8e6-19fa-4d9b-a816-5b1425172932"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:40:27.885857 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.885716 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "99fbf8e6-19fa-4d9b-a816-5b1425172932" (UID: "99fbf8e6-19fa-4d9b-a816-5b1425172932"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:40:27.983643 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.983605 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2m27\" (UniqueName: \"kubernetes.io/projected/99fbf8e6-19fa-4d9b-a816-5b1425172932-kube-api-access-h2m27\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:40:27.983643 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.983638 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99fbf8e6-19fa-4d9b-a816-5b1425172932-istiod-ca-cert\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:40:27.983643 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.983651 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-workload-certs\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:40:27.983921 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.983663 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-podinfo\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:40:27.983921 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.983674 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-envoy\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:40:27.983921 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.983685 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-workload-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:40:27.983921 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.983697 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-credential-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:40:27.983921 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.983707 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-data\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:40:27.983921 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:27.983718 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99fbf8e6-19fa-4d9b-a816-5b1425172932-istio-token\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:40:28.045776 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:28.045743 2583 generic.go:356] "Generic (PLEG): container finished" podID="99fbf8e6-19fa-4d9b-a816-5b1425172932" containerID="b4c5d24b3a15bf455895e88d59389ac48c8eeae6f14998def7e49a54e54fd949" exitCode=0 Jul 02 23:40:28.046009 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:28.045786 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" event={"ID":"99fbf8e6-19fa-4d9b-a816-5b1425172932","Type":"ContainerDied","Data":"b4c5d24b3a15bf455895e88d59389ac48c8eeae6f14998def7e49a54e54fd949"} Jul 02 23:40:28.046009 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:28.045803 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" Jul 02 23:40:28.046009 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:28.045827 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf" event={"ID":"99fbf8e6-19fa-4d9b-a816-5b1425172932","Type":"ContainerDied","Data":"2abbdb29951ef46ddda2a95a3e07b765e8369f48bc7ed28d790241b3a8d67212"} Jul 02 23:40:28.046009 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:28.045849 2583 scope.go:117] "RemoveContainer" containerID="b4c5d24b3a15bf455895e88d59389ac48c8eeae6f14998def7e49a54e54fd949" Jul 02 23:40:28.055339 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:28.055315 2583 scope.go:117] "RemoveContainer" containerID="06883aea9c9822f91769a79e1b89b36999e37079b697c32a911cf8d4393a8eaf" Jul 02 23:40:28.063386 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:28.063369 2583 scope.go:117] "RemoveContainer" containerID="b4c5d24b3a15bf455895e88d59389ac48c8eeae6f14998def7e49a54e54fd949" Jul 02 23:40:28.063675 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:40:28.063655 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4c5d24b3a15bf455895e88d59389ac48c8eeae6f14998def7e49a54e54fd949\": container with ID starting with b4c5d24b3a15bf455895e88d59389ac48c8eeae6f14998def7e49a54e54fd949 not found: ID does not exist" containerID="b4c5d24b3a15bf455895e88d59389ac48c8eeae6f14998def7e49a54e54fd949" Jul 02 23:40:28.063746 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:28.063688 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4c5d24b3a15bf455895e88d59389ac48c8eeae6f14998def7e49a54e54fd949"} err="failed to get container status \"b4c5d24b3a15bf455895e88d59389ac48c8eeae6f14998def7e49a54e54fd949\": rpc error: code = NotFound desc = could not find container \"b4c5d24b3a15bf455895e88d59389ac48c8eeae6f14998def7e49a54e54fd949\": container with ID starting with b4c5d24b3a15bf455895e88d59389ac48c8eeae6f14998def7e49a54e54fd949 not found: ID does not exist" Jul 02 23:40:28.063746 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:28.063714 2583 scope.go:117] "RemoveContainer" containerID="06883aea9c9822f91769a79e1b89b36999e37079b697c32a911cf8d4393a8eaf" Jul 02 23:40:28.064022 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:40:28.064004 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06883aea9c9822f91769a79e1b89b36999e37079b697c32a911cf8d4393a8eaf\": container with ID starting with 06883aea9c9822f91769a79e1b89b36999e37079b697c32a911cf8d4393a8eaf not found: ID does not exist" containerID="06883aea9c9822f91769a79e1b89b36999e37079b697c32a911cf8d4393a8eaf" Jul 02 23:40:28.064072 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:28.064028 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06883aea9c9822f91769a79e1b89b36999e37079b697c32a911cf8d4393a8eaf"} err="failed to get container status \"06883aea9c9822f91769a79e1b89b36999e37079b697c32a911cf8d4393a8eaf\": rpc error: code = NotFound desc = could not find container \"06883aea9c9822f91769a79e1b89b36999e37079b697c32a911cf8d4393a8eaf\": container with ID starting with 06883aea9c9822f91769a79e1b89b36999e37079b697c32a911cf8d4393a8eaf not found: ID does not exist" Jul 02 23:40:28.068134 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:28.068110 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf"] Jul 02 23:40:28.073327 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:28.073303 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-979f9f-openshift-default-7c4d8c6dbc-lv8mf"] Jul 02 23:40:28.532703 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:28.532657 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99fbf8e6-19fa-4d9b-a816-5b1425172932" path="/var/lib/kubelet/pods/99fbf8e6-19fa-4d9b-a816-5b1425172932/volumes" Jul 02 23:40:29.268636 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:29.268599 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg"] Jul 02 23:40:29.269053 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:29.268840 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" podUID="15ef0459-de5d-4547-93c8-679e0009e53a" containerName="istio-proxy" containerID="cri-o://22c08777b7a2ddfcc66e58072a8a4a72f372ed5f4fb16012c17d17b189541d6c" gracePeriod=30 Jul 02 23:40:30.118055 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.118021 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt"] Jul 02 23:40:30.118415 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.118402 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99fbf8e6-19fa-4d9b-a816-5b1425172932" containerName="istio-proxy" Jul 02 23:40:30.118469 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.118416 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="99fbf8e6-19fa-4d9b-a816-5b1425172932" containerName="istio-proxy" Jul 02 23:40:30.118469 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.118429 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99fbf8e6-19fa-4d9b-a816-5b1425172932" containerName="istio-proxy" Jul 02 23:40:30.118469 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.118434 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="99fbf8e6-19fa-4d9b-a816-5b1425172932" containerName="istio-proxy" Jul 02 23:40:30.118565 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.118510 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="99fbf8e6-19fa-4d9b-a816-5b1425172932" containerName="istio-proxy" Jul 02 23:40:30.121484 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.121460 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.123807 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.123783 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-115c59-openshift-default-dockercfg-pcmm6\"" Jul 02 23:40:30.131016 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.130972 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt"] Jul 02 23:40:30.205173 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.205131 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-istio-data\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.205392 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.205208 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-credential-socket\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.205392 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.205237 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/521c6dd9-fc83-412d-ac67-afadffde3488-istio-token\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.205392 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.205330 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-workload-socket\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.205392 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.205367 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-workload-certs\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.205596 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.205397 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-istio-envoy\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.205596 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.205425 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/521c6dd9-fc83-412d-ac67-afadffde3488-istio-podinfo\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.205596 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.205451 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4nvf\" (UniqueName: \"kubernetes.io/projected/521c6dd9-fc83-412d-ac67-afadffde3488-kube-api-access-v4nvf\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.205596 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.205480 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/521c6dd9-fc83-412d-ac67-afadffde3488-istiod-ca-cert\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.306584 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.306531 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-istio-data\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.306584 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.306594 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-credential-socket\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.307125 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.306621 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/521c6dd9-fc83-412d-ac67-afadffde3488-istio-token\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.307125 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.306679 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-workload-socket\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.307125 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.306819 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-workload-certs\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.307125 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.306876 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-istio-envoy\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.307125 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.306923 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/521c6dd9-fc83-412d-ac67-afadffde3488-istio-podinfo\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.307125 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.306951 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4nvf\" (UniqueName: \"kubernetes.io/projected/521c6dd9-fc83-412d-ac67-afadffde3488-kube-api-access-v4nvf\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.307125 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.306983 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/521c6dd9-fc83-412d-ac67-afadffde3488-istiod-ca-cert\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.307125 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.306988 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-istio-data\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.307125 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.307047 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-workload-socket\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.307567 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.307154 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-credential-socket\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.307567 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.307205 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-workload-certs\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.307679 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.307657 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/521c6dd9-fc83-412d-ac67-afadffde3488-istiod-ca-cert\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.309328 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.309305 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-istio-envoy\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.309641 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.309621 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/521c6dd9-fc83-412d-ac67-afadffde3488-istio-podinfo\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.314656 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.314633 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/521c6dd9-fc83-412d-ac67-afadffde3488-istio-token\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.314795 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.314776 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4nvf\" (UniqueName: \"kubernetes.io/projected/521c6dd9-fc83-412d-ac67-afadffde3488-kube-api-access-v4nvf\") pod \"e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.432645 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.432551 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:30.584133 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.584102 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt"] Jul 02 23:40:30.586839 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:40:30.586812 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod521c6dd9_fc83_412d_ac67_afadffde3488.slice/crio-e97dc294ddc6db987527d3fc7debe7c24279dcb491e7aa0635e3d3fb5d4d5284 WatchSource:0}: Error finding container e97dc294ddc6db987527d3fc7debe7c24279dcb491e7aa0635e3d3fb5d4d5284: Status 404 returned error can't find the container with id e97dc294ddc6db987527d3fc7debe7c24279dcb491e7aa0635e3d3fb5d4d5284 Jul 02 23:40:30.588989 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.588956 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:30.589089 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.589025 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:30.589089 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:30.589055 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:31.069365 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:31.069326 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" event={"ID":"521c6dd9-fc83-412d-ac67-afadffde3488","Type":"ContainerStarted","Data":"400b66381b29dfccf3844aec9dd30860ca2b2e30a071019f32c693e97caf2ae0"} Jul 02 23:40:31.069549 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:31.069379 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" event={"ID":"521c6dd9-fc83-412d-ac67-afadffde3488","Type":"ContainerStarted","Data":"e97dc294ddc6db987527d3fc7debe7c24279dcb491e7aa0635e3d3fb5d4d5284"} Jul 02 23:40:31.088287 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:31.088237 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" podStartSLOduration=1.088221598 podStartE2EDuration="1.088221598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:40:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:40:31.086798633 +0000 UTC m=+1995.109616746" watchObservedRunningTime="2026-07-02 23:40:31.088221598 +0000 UTC m=+1995.111039767" Jul 02 23:40:31.433414 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:31.433328 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:31.438421 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:31.438396 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:32.073965 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:32.073934 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:32.075010 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:32.074986 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:34.527282 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.527249 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:34.645637 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.645605 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15ef0459-de5d-4547-93c8-679e0009e53a-istiod-ca-cert\") pod \"15ef0459-de5d-4547-93c8-679e0009e53a\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " Jul 02 23:40:34.645847 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.645650 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15ef0459-de5d-4547-93c8-679e0009e53a-istio-token\") pod \"15ef0459-de5d-4547-93c8-679e0009e53a\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " Jul 02 23:40:34.645847 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.645719 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15ef0459-de5d-4547-93c8-679e0009e53a-istio-podinfo\") pod \"15ef0459-de5d-4547-93c8-679e0009e53a\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " Jul 02 23:40:34.645847 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.645743 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-istio-envoy\") pod \"15ef0459-de5d-4547-93c8-679e0009e53a\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " Jul 02 23:40:34.646062 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.645924 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-credential-socket\") pod \"15ef0459-de5d-4547-93c8-679e0009e53a\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " Jul 02 23:40:34.646062 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.645967 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-istio-data\") pod \"15ef0459-de5d-4547-93c8-679e0009e53a\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " Jul 02 23:40:34.646062 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.646016 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-workload-socket\") pod \"15ef0459-de5d-4547-93c8-679e0009e53a\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " Jul 02 23:40:34.646062 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.646054 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qv8g\" (UniqueName: \"kubernetes.io/projected/15ef0459-de5d-4547-93c8-679e0009e53a-kube-api-access-7qv8g\") pod \"15ef0459-de5d-4547-93c8-679e0009e53a\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " Jul 02 23:40:34.646062 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.646053 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15ef0459-de5d-4547-93c8-679e0009e53a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "15ef0459-de5d-4547-93c8-679e0009e53a" (UID: "15ef0459-de5d-4547-93c8-679e0009e53a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:40:34.646310 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.646114 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-workload-certs\") pod \"15ef0459-de5d-4547-93c8-679e0009e53a\" (UID: \"15ef0459-de5d-4547-93c8-679e0009e53a\") " Jul 02 23:40:34.646706 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.646478 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "15ef0459-de5d-4547-93c8-679e0009e53a" (UID: "15ef0459-de5d-4547-93c8-679e0009e53a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:40:34.646706 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.646638 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "15ef0459-de5d-4547-93c8-679e0009e53a" (UID: "15ef0459-de5d-4547-93c8-679e0009e53a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:40:34.646883 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.646643 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15ef0459-de5d-4547-93c8-679e0009e53a-istiod-ca-cert\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:40:34.646883 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.646781 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-credential-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:40:34.646883 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.646689 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "15ef0459-de5d-4547-93c8-679e0009e53a" (UID: "15ef0459-de5d-4547-93c8-679e0009e53a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:40:34.647077 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.646977 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-istio-data" (OuterVolumeSpecName: "istio-data") pod "15ef0459-de5d-4547-93c8-679e0009e53a" (UID: "15ef0459-de5d-4547-93c8-679e0009e53a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:40:34.648300 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.648273 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/15ef0459-de5d-4547-93c8-679e0009e53a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "15ef0459-de5d-4547-93c8-679e0009e53a" (UID: "15ef0459-de5d-4547-93c8-679e0009e53a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:40:34.648396 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.648292 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15ef0459-de5d-4547-93c8-679e0009e53a-kube-api-access-7qv8g" (OuterVolumeSpecName: "kube-api-access-7qv8g") pod "15ef0459-de5d-4547-93c8-679e0009e53a" (UID: "15ef0459-de5d-4547-93c8-679e0009e53a"). InnerVolumeSpecName "kube-api-access-7qv8g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:40:34.648604 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.648586 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "15ef0459-de5d-4547-93c8-679e0009e53a" (UID: "15ef0459-de5d-4547-93c8-679e0009e53a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:40:34.648664 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.648605 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15ef0459-de5d-4547-93c8-679e0009e53a-istio-token" (OuterVolumeSpecName: "istio-token") pod "15ef0459-de5d-4547-93c8-679e0009e53a" (UID: "15ef0459-de5d-4547-93c8-679e0009e53a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:40:34.747546 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.747506 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-workload-certs\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:40:34.747546 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.747541 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15ef0459-de5d-4547-93c8-679e0009e53a-istio-token\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:40:34.747546 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.747550 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15ef0459-de5d-4547-93c8-679e0009e53a-istio-podinfo\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:40:34.747546 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.747557 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-istio-envoy\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:40:34.747804 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.747566 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-istio-data\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:40:34.747804 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.747574 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15ef0459-de5d-4547-93c8-679e0009e53a-workload-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:40:34.747804 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:34.747582 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qv8g\" (UniqueName: \"kubernetes.io/projected/15ef0459-de5d-4547-93c8-679e0009e53a-kube-api-access-7qv8g\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:40:35.095501 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:35.095468 2583 generic.go:356] "Generic (PLEG): container finished" podID="15ef0459-de5d-4547-93c8-679e0009e53a" containerID="22c08777b7a2ddfcc66e58072a8a4a72f372ed5f4fb16012c17d17b189541d6c" exitCode=0 Jul 02 23:40:35.095683 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:35.095518 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" Jul 02 23:40:35.095683 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:35.095566 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" event={"ID":"15ef0459-de5d-4547-93c8-679e0009e53a","Type":"ContainerDied","Data":"22c08777b7a2ddfcc66e58072a8a4a72f372ed5f4fb16012c17d17b189541d6c"} Jul 02 23:40:35.095683 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:35.095623 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg" event={"ID":"15ef0459-de5d-4547-93c8-679e0009e53a","Type":"ContainerDied","Data":"1233afe6a775fc00aca8a5b59e58f54e30ed9b43b88ce377a458d264d0945e13"} Jul 02 23:40:35.095683 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:35.095651 2583 scope.go:117] "RemoveContainer" containerID="22c08777b7a2ddfcc66e58072a8a4a72f372ed5f4fb16012c17d17b189541d6c" Jul 02 23:40:35.104554 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:35.104538 2583 scope.go:117] "RemoveContainer" containerID="640d55c53440cc6bd3708c2d957679bec5f86bbf86512e7d51726244ed99456e" Jul 02 23:40:35.112362 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:35.112338 2583 scope.go:117] "RemoveContainer" containerID="22c08777b7a2ddfcc66e58072a8a4a72f372ed5f4fb16012c17d17b189541d6c" Jul 02 23:40:35.112614 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:40:35.112595 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22c08777b7a2ddfcc66e58072a8a4a72f372ed5f4fb16012c17d17b189541d6c\": container with ID starting with 22c08777b7a2ddfcc66e58072a8a4a72f372ed5f4fb16012c17d17b189541d6c not found: ID does not exist" containerID="22c08777b7a2ddfcc66e58072a8a4a72f372ed5f4fb16012c17d17b189541d6c" Jul 02 23:40:35.112693 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:35.112626 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22c08777b7a2ddfcc66e58072a8a4a72f372ed5f4fb16012c17d17b189541d6c"} err="failed to get container status \"22c08777b7a2ddfcc66e58072a8a4a72f372ed5f4fb16012c17d17b189541d6c\": rpc error: code = NotFound desc = could not find container \"22c08777b7a2ddfcc66e58072a8a4a72f372ed5f4fb16012c17d17b189541d6c\": container with ID starting with 22c08777b7a2ddfcc66e58072a8a4a72f372ed5f4fb16012c17d17b189541d6c not found: ID does not exist" Jul 02 23:40:35.112693 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:35.112651 2583 scope.go:117] "RemoveContainer" containerID="640d55c53440cc6bd3708c2d957679bec5f86bbf86512e7d51726244ed99456e" Jul 02 23:40:35.112911 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:40:35.112869 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"640d55c53440cc6bd3708c2d957679bec5f86bbf86512e7d51726244ed99456e\": container with ID starting with 640d55c53440cc6bd3708c2d957679bec5f86bbf86512e7d51726244ed99456e not found: ID does not exist" containerID="640d55c53440cc6bd3708c2d957679bec5f86bbf86512e7d51726244ed99456e" Jul 02 23:40:35.112960 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:35.112942 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"640d55c53440cc6bd3708c2d957679bec5f86bbf86512e7d51726244ed99456e"} err="failed to get container status \"640d55c53440cc6bd3708c2d957679bec5f86bbf86512e7d51726244ed99456e\": rpc error: code = NotFound desc = could not find container \"640d55c53440cc6bd3708c2d957679bec5f86bbf86512e7d51726244ed99456e\": container with ID starting with 640d55c53440cc6bd3708c2d957679bec5f86bbf86512e7d51726244ed99456e not found: ID does not exist" Jul 02 23:40:35.115691 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:35.115668 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg"] Jul 02 23:40:35.119445 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:35.119419 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-6c2ab8-openshift-default-685d896445-dz5jg"] Jul 02 23:40:36.532854 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:36.532820 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15ef0459-de5d-4547-93c8-679e0009e53a" path="/var/lib/kubelet/pods/15ef0459-de5d-4547-93c8-679e0009e53a/volumes" Jul 02 23:40:38.126688 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.126655 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg"] Jul 02 23:40:38.127072 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.127036 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15ef0459-de5d-4547-93c8-679e0009e53a" containerName="istio-proxy" Jul 02 23:40:38.127072 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.127051 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="15ef0459-de5d-4547-93c8-679e0009e53a" containerName="istio-proxy" Jul 02 23:40:38.127072 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.127071 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15ef0459-de5d-4547-93c8-679e0009e53a" containerName="istio-proxy" Jul 02 23:40:38.127219 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.127081 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="15ef0459-de5d-4547-93c8-679e0009e53a" containerName="istio-proxy" Jul 02 23:40:38.127219 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.127171 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="99fbf8e6-19fa-4d9b-a816-5b1425172932" containerName="istio-proxy" Jul 02 23:40:38.127219 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.127184 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="15ef0459-de5d-4547-93c8-679e0009e53a" containerName="istio-proxy" Jul 02 23:40:38.130037 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.130015 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.132140 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.132115 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-f1cae1-openshift-default-dockercfg-h7879\"" Jul 02 23:40:38.139992 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.139973 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg"] Jul 02 23:40:38.279833 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.279794 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-workload-socket\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.279833 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.279834 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsc7f\" (UniqueName: \"kubernetes.io/projected/5037d4d9-6c79-421c-bd81-4b2f160f0d71-kube-api-access-vsc7f\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.280075 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.279868 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-credential-socket\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.280075 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.279980 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-workload-certs\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.280075 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.280011 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-envoy\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.280075 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.280043 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istiod-ca-cert\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.280212 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.280084 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-data\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.280212 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.280100 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-token\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.280212 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.280118 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-podinfo\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.381109 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.381013 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-data\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.381109 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.381056 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-token\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.381109 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.381076 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-podinfo\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.381109 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.381104 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-workload-socket\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.381418 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.381132 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsc7f\" (UniqueName: \"kubernetes.io/projected/5037d4d9-6c79-421c-bd81-4b2f160f0d71-kube-api-access-vsc7f\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.381418 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.381169 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-credential-socket\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.381418 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.381245 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-workload-certs\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.381418 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.381272 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-envoy\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.381418 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.381310 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istiod-ca-cert\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.381692 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.381479 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-data\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.381692 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.381530 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-workload-socket\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.381692 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.381638 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-credential-socket\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.381805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.381688 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-workload-certs\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.381882 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.381863 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istiod-ca-cert\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.383656 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.383632 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-envoy\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.383769 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.383717 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-podinfo\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.389420 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.389394 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsc7f\" (UniqueName: \"kubernetes.io/projected/5037d4d9-6c79-421c-bd81-4b2f160f0d71-kube-api-access-vsc7f\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.389420 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.389406 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-token\") pod \"e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.441468 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.441429 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:38.583672 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.583642 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg"] Jul 02 23:40:38.586449 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:40:38.586421 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5037d4d9_6c79_421c_bd81_4b2f160f0d71.slice/crio-62418c5a455facde893da01d3d7bee72ed57c449e9c570864f491d1fa7ab6852 WatchSource:0}: Error finding container 62418c5a455facde893da01d3d7bee72ed57c449e9c570864f491d1fa7ab6852: Status 404 returned error can't find the container with id 62418c5a455facde893da01d3d7bee72ed57c449e9c570864f491d1fa7ab6852 Jul 02 23:40:38.588968 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.588936 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:38.589073 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.589015 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:38.589132 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:38.589088 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:39.118939 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:39.118873 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" event={"ID":"5037d4d9-6c79-421c-bd81-4b2f160f0d71","Type":"ContainerStarted","Data":"45f6b36c48dda0e75389fab6c7e9dc1a6c57559b22c3fe2a1a6612716a11c706"} Jul 02 23:40:39.119158 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:39.118990 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" event={"ID":"5037d4d9-6c79-421c-bd81-4b2f160f0d71","Type":"ContainerStarted","Data":"62418c5a455facde893da01d3d7bee72ed57c449e9c570864f491d1fa7ab6852"} Jul 02 23:40:39.138649 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:39.138598 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" podStartSLOduration=1.138583735 podStartE2EDuration="1.138583735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:40:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:40:39.135795616 +0000 UTC m=+2003.158613729" watchObservedRunningTime="2026-07-02 23:40:39.138583735 +0000 UTC m=+2003.161401847" Jul 02 23:40:39.441700 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:39.441601 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:39.446992 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:39.446963 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:40.123652 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:40.123621 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:40.124562 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:40.124537 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:47.300123 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:47.300081 2583 patch_prober.go:28] interesting pod/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": read tcp 10.133.0.2:35636->10.133.0.55:15021: read: connection reset by peer" start-of-body= Jul 02 23:40:47.300535 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:47.300166 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" podUID="521c6dd9-fc83-412d-ac67-afadffde3488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": read tcp 10.133.0.2:35636->10.133.0.55:15021: read: connection reset by peer" Jul 02 23:40:48.164416 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:48.164383 2583 generic.go:356] "Generic (PLEG): container finished" podID="521c6dd9-fc83-412d-ac67-afadffde3488" containerID="400b66381b29dfccf3844aec9dd30860ca2b2e30a071019f32c693e97caf2ae0" exitCode=0 Jul 02 23:40:48.164610 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:48.164440 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" event={"ID":"521c6dd9-fc83-412d-ac67-afadffde3488","Type":"ContainerDied","Data":"400b66381b29dfccf3844aec9dd30860ca2b2e30a071019f32c693e97caf2ae0"} Jul 02 23:40:48.164824 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:48.164810 2583 scope.go:117] "RemoveContainer" containerID="400b66381b29dfccf3844aec9dd30860ca2b2e30a071019f32c693e97caf2ae0" Jul 02 23:40:48.165796 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:48.165768 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:48.165846 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:48.165838 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:48.165884 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:48.165870 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:49.170650 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:49.170614 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" event={"ID":"521c6dd9-fc83-412d-ac67-afadffde3488","Type":"ContainerStarted","Data":"3cb5f709cbf8142b5bc56fab5d16562c8ee43a034c0726dd9243f702456d6e00"} Jul 02 23:40:49.433267 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:49.433163 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:49.433450 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:49.433422 2583 patch_prober.go:28] interesting pod/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 02 23:40:49.433538 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:49.433467 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" podUID="521c6dd9-fc83-412d-ac67-afadffde3488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 02 23:40:50.433067 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:50.433023 2583 patch_prober.go:28] interesting pod/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 02 23:40:50.433582 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:50.433096 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" podUID="521c6dd9-fc83-412d-ac67-afadffde3488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 02 23:40:51.433756 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:51.433718 2583 patch_prober.go:28] interesting pod/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 02 23:40:51.434245 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:51.433781 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" podUID="521c6dd9-fc83-412d-ac67-afadffde3488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 02 23:40:52.433092 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:52.433052 2583 patch_prober.go:28] interesting pod/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 02 23:40:52.433287 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:52.433143 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" podUID="521c6dd9-fc83-412d-ac67-afadffde3488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 02 23:40:53.201034 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:53.200997 2583 generic.go:356] "Generic (PLEG): container finished" podID="5037d4d9-6c79-421c-bd81-4b2f160f0d71" containerID="45f6b36c48dda0e75389fab6c7e9dc1a6c57559b22c3fe2a1a6612716a11c706" exitCode=0 Jul 02 23:40:53.201500 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:53.201091 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" event={"ID":"5037d4d9-6c79-421c-bd81-4b2f160f0d71","Type":"ContainerDied","Data":"45f6b36c48dda0e75389fab6c7e9dc1a6c57559b22c3fe2a1a6612716a11c706"} Jul 02 23:40:53.201598 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:53.201582 2583 scope.go:117] "RemoveContainer" containerID="45f6b36c48dda0e75389fab6c7e9dc1a6c57559b22c3fe2a1a6612716a11c706" Jul 02 23:40:53.202606 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:53.202579 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:53.202719 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:53.202644 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:53.202719 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:53.202679 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 23:40:53.436734 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:53.436698 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:53.437161 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:53.437134 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:53.437962 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:53.437943 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:40:54.208357 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:54.208320 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" event={"ID":"5037d4d9-6c79-421c-bd81-4b2f160f0d71","Type":"ContainerStarted","Data":"88998b22e30ba7dfe5ffe01c57e105655000ebea868a4f4adf653813f2b76dc2"} Jul 02 23:40:54.442424 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:54.442386 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:54.443651 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:54.443619 2583 patch_prober.go:28] interesting pod/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 02 23:40:54.443713 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:54.443678 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" podUID="5037d4d9-6c79-421c-bd81-4b2f160f0d71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 02 23:40:55.124175 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:55.124129 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:55.442766 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:55.442675 2583 patch_prober.go:28] interesting pod/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 02 23:40:55.442766 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:55.442731 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" podUID="5037d4d9-6c79-421c-bd81-4b2f160f0d71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 02 23:40:56.442301 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:56.442262 2583 patch_prober.go:28] interesting pod/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 02 23:40:56.442495 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:56.442346 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" podUID="5037d4d9-6c79-421c-bd81-4b2f160f0d71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 02 23:40:57.442694 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:57.442651 2583 patch_prober.go:28] interesting pod/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 02 23:40:57.443168 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:57.442710 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" podUID="5037d4d9-6c79-421c-bd81-4b2f160f0d71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 02 23:40:58.446290 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:58.446261 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:40:58.447183 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:40:58.447164 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:43:56.576417 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:43:56.576377 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt"] Jul 02 23:43:56.576977 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:43:56.576660 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" podUID="521c6dd9-fc83-412d-ac67-afadffde3488" containerName="istio-proxy" containerID="cri-o://3cb5f709cbf8142b5bc56fab5d16562c8ee43a034c0726dd9243f702456d6e00" gracePeriod=30 Jul 02 23:44:01.832671 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.832645 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:44:01.947661 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.947552 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/521c6dd9-fc83-412d-ac67-afadffde3488-istiod-ca-cert\") pod \"521c6dd9-fc83-412d-ac67-afadffde3488\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " Jul 02 23:44:01.947661 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.947608 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/521c6dd9-fc83-412d-ac67-afadffde3488-istio-podinfo\") pod \"521c6dd9-fc83-412d-ac67-afadffde3488\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " Jul 02 23:44:01.947661 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.947633 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-workload-certs\") pod \"521c6dd9-fc83-412d-ac67-afadffde3488\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " Jul 02 23:44:01.947661 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.947664 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-workload-socket\") pod \"521c6dd9-fc83-412d-ac67-afadffde3488\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " Jul 02 23:44:01.948047 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.947704 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-istio-envoy\") pod \"521c6dd9-fc83-412d-ac67-afadffde3488\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " Jul 02 23:44:01.948047 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.947744 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/521c6dd9-fc83-412d-ac67-afadffde3488-istio-token\") pod \"521c6dd9-fc83-412d-ac67-afadffde3488\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " Jul 02 23:44:01.948047 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.947769 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4nvf\" (UniqueName: \"kubernetes.io/projected/521c6dd9-fc83-412d-ac67-afadffde3488-kube-api-access-v4nvf\") pod \"521c6dd9-fc83-412d-ac67-afadffde3488\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " Jul 02 23:44:01.948047 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.947803 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-credential-socket\") pod \"521c6dd9-fc83-412d-ac67-afadffde3488\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " Jul 02 23:44:01.948047 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.947834 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-istio-data\") pod \"521c6dd9-fc83-412d-ac67-afadffde3488\" (UID: \"521c6dd9-fc83-412d-ac67-afadffde3488\") " Jul 02 23:44:01.948317 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.948060 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "521c6dd9-fc83-412d-ac67-afadffde3488" (UID: "521c6dd9-fc83-412d-ac67-afadffde3488"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:44:01.948317 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.948087 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "521c6dd9-fc83-412d-ac67-afadffde3488" (UID: "521c6dd9-fc83-412d-ac67-afadffde3488"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:44:01.948317 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.948294 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "521c6dd9-fc83-412d-ac67-afadffde3488" (UID: "521c6dd9-fc83-412d-ac67-afadffde3488"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:44:01.948481 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.948376 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-credential-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:44:01.948481 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.948391 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-workload-certs\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:44:01.948481 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.948404 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-workload-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:44:01.948481 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.948407 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/521c6dd9-fc83-412d-ac67-afadffde3488-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "521c6dd9-fc83-412d-ac67-afadffde3488" (UID: "521c6dd9-fc83-412d-ac67-afadffde3488"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:44:01.948868 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.948843 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-istio-data" (OuterVolumeSpecName: "istio-data") pod "521c6dd9-fc83-412d-ac67-afadffde3488" (UID: "521c6dd9-fc83-412d-ac67-afadffde3488"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:44:01.950214 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.950185 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "521c6dd9-fc83-412d-ac67-afadffde3488" (UID: "521c6dd9-fc83-412d-ac67-afadffde3488"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:44:01.950360 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.950309 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/521c6dd9-fc83-412d-ac67-afadffde3488-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "521c6dd9-fc83-412d-ac67-afadffde3488" (UID: "521c6dd9-fc83-412d-ac67-afadffde3488"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:44:01.950360 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.950337 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/521c6dd9-fc83-412d-ac67-afadffde3488-istio-token" (OuterVolumeSpecName: "istio-token") pod "521c6dd9-fc83-412d-ac67-afadffde3488" (UID: "521c6dd9-fc83-412d-ac67-afadffde3488"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:44:01.950463 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:01.950438 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/521c6dd9-fc83-412d-ac67-afadffde3488-kube-api-access-v4nvf" (OuterVolumeSpecName: "kube-api-access-v4nvf") pod "521c6dd9-fc83-412d-ac67-afadffde3488" (UID: "521c6dd9-fc83-412d-ac67-afadffde3488"). InnerVolumeSpecName "kube-api-access-v4nvf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:44:02.048842 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:02.048804 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-istio-envoy\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:44:02.048842 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:02.048835 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/521c6dd9-fc83-412d-ac67-afadffde3488-istio-token\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:44:02.048842 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:02.048845 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4nvf\" (UniqueName: \"kubernetes.io/projected/521c6dd9-fc83-412d-ac67-afadffde3488-kube-api-access-v4nvf\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:44:02.048842 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:02.048855 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/521c6dd9-fc83-412d-ac67-afadffde3488-istio-data\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:44:02.049137 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:02.048863 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/521c6dd9-fc83-412d-ac67-afadffde3488-istiod-ca-cert\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:44:02.049137 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:02.048870 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/521c6dd9-fc83-412d-ac67-afadffde3488-istio-podinfo\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:44:02.124426 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:02.124389 2583 generic.go:356] "Generic (PLEG): container finished" podID="521c6dd9-fc83-412d-ac67-afadffde3488" containerID="3cb5f709cbf8142b5bc56fab5d16562c8ee43a034c0726dd9243f702456d6e00" exitCode=0 Jul 02 23:44:02.124612 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:02.124436 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" Jul 02 23:44:02.124612 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:02.124479 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" event={"ID":"521c6dd9-fc83-412d-ac67-afadffde3488","Type":"ContainerDied","Data":"3cb5f709cbf8142b5bc56fab5d16562c8ee43a034c0726dd9243f702456d6e00"} Jul 02 23:44:02.124612 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:02.124542 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt" event={"ID":"521c6dd9-fc83-412d-ac67-afadffde3488","Type":"ContainerDied","Data":"e97dc294ddc6db987527d3fc7debe7c24279dcb491e7aa0635e3d3fb5d4d5284"} Jul 02 23:44:02.124612 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:02.124572 2583 scope.go:117] "RemoveContainer" containerID="3cb5f709cbf8142b5bc56fab5d16562c8ee43a034c0726dd9243f702456d6e00" Jul 02 23:44:02.134162 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:02.134144 2583 scope.go:117] "RemoveContainer" containerID="400b66381b29dfccf3844aec9dd30860ca2b2e30a071019f32c693e97caf2ae0" Jul 02 23:44:02.142475 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:02.142447 2583 scope.go:117] "RemoveContainer" containerID="3cb5f709cbf8142b5bc56fab5d16562c8ee43a034c0726dd9243f702456d6e00" Jul 02 23:44:02.142833 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:44:02.142806 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cb5f709cbf8142b5bc56fab5d16562c8ee43a034c0726dd9243f702456d6e00\": container with ID starting with 3cb5f709cbf8142b5bc56fab5d16562c8ee43a034c0726dd9243f702456d6e00 not found: ID does not exist" containerID="3cb5f709cbf8142b5bc56fab5d16562c8ee43a034c0726dd9243f702456d6e00" Jul 02 23:44:02.143005 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:02.142839 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cb5f709cbf8142b5bc56fab5d16562c8ee43a034c0726dd9243f702456d6e00"} err="failed to get container status \"3cb5f709cbf8142b5bc56fab5d16562c8ee43a034c0726dd9243f702456d6e00\": rpc error: code = NotFound desc = could not find container \"3cb5f709cbf8142b5bc56fab5d16562c8ee43a034c0726dd9243f702456d6e00\": container with ID starting with 3cb5f709cbf8142b5bc56fab5d16562c8ee43a034c0726dd9243f702456d6e00 not found: ID does not exist" Jul 02 23:44:02.143005 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:02.142860 2583 scope.go:117] "RemoveContainer" containerID="400b66381b29dfccf3844aec9dd30860ca2b2e30a071019f32c693e97caf2ae0" Jul 02 23:44:02.143265 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:44:02.143240 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"400b66381b29dfccf3844aec9dd30860ca2b2e30a071019f32c693e97caf2ae0\": container with ID starting with 400b66381b29dfccf3844aec9dd30860ca2b2e30a071019f32c693e97caf2ae0 not found: ID does not exist" containerID="400b66381b29dfccf3844aec9dd30860ca2b2e30a071019f32c693e97caf2ae0" Jul 02 23:44:02.143321 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:02.143273 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"400b66381b29dfccf3844aec9dd30860ca2b2e30a071019f32c693e97caf2ae0"} err="failed to get container status \"400b66381b29dfccf3844aec9dd30860ca2b2e30a071019f32c693e97caf2ae0\": rpc error: code = NotFound desc = could not find container \"400b66381b29dfccf3844aec9dd30860ca2b2e30a071019f32c693e97caf2ae0\": container with ID starting with 400b66381b29dfccf3844aec9dd30860ca2b2e30a071019f32c693e97caf2ae0 not found: ID does not exist" Jul 02 23:44:02.145533 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:02.145513 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt"] Jul 02 23:44:02.148557 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:02.148536 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-115c59-openshift-default-787bf4b67-mkddt"] Jul 02 23:44:02.533297 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:02.533264 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="521c6dd9-fc83-412d-ac67-afadffde3488" path="/var/lib/kubelet/pods/521c6dd9-fc83-412d-ac67-afadffde3488/volumes" Jul 02 23:44:03.387199 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:03.387165 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg"] Jul 02 23:44:03.387668 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:03.387415 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" podUID="5037d4d9-6c79-421c-bd81-4b2f160f0d71" containerName="istio-proxy" containerID="cri-o://88998b22e30ba7dfe5ffe01c57e105655000ebea868a4f4adf653813f2b76dc2" gracePeriod=30 Jul 02 23:44:04.135826 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:04.135792 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5b754c5d8d-vql6w_d2258cf2-7688-4cd2-ade2-01163ffd216a/authorino/0.log" Jul 02 23:44:08.625824 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.625795 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-5nccz_3dc2f7e3-a8b3-4280-a2f1-828b24282c70/manager/0.log" Jul 02 23:44:08.645122 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.645098 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:44:08.696934 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.696824 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-envoy\") pod \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " Jul 02 23:44:08.696934 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.696861 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istiod-ca-cert\") pod \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " Jul 02 23:44:08.696934 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.696881 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-podinfo\") pod \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " Jul 02 23:44:08.697190 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.696942 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vsc7f\" (UniqueName: \"kubernetes.io/projected/5037d4d9-6c79-421c-bd81-4b2f160f0d71-kube-api-access-vsc7f\") pod \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " Jul 02 23:44:08.697190 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.696981 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-credential-socket\") pod \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " Jul 02 23:44:08.697190 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.696995 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-data\") pod \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " Jul 02 23:44:08.697190 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.697023 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-workload-certs\") pod \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " Jul 02 23:44:08.697190 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.697086 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-token\") pod \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " Jul 02 23:44:08.697190 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.697111 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-workload-socket\") pod \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\" (UID: \"5037d4d9-6c79-421c-bd81-4b2f160f0d71\") " Jul 02 23:44:08.697564 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.697512 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5037d4d9-6c79-421c-bd81-4b2f160f0d71" (UID: "5037d4d9-6c79-421c-bd81-4b2f160f0d71"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:44:08.697564 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.697526 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5037d4d9-6c79-421c-bd81-4b2f160f0d71" (UID: "5037d4d9-6c79-421c-bd81-4b2f160f0d71"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:44:08.697716 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.697695 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5037d4d9-6c79-421c-bd81-4b2f160f0d71" (UID: "5037d4d9-6c79-421c-bd81-4b2f160f0d71"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:44:08.697779 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.697758 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5037d4d9-6c79-421c-bd81-4b2f160f0d71" (UID: "5037d4d9-6c79-421c-bd81-4b2f160f0d71"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:44:08.698786 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.698755 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-data" (OuterVolumeSpecName: "istio-data") pod "5037d4d9-6c79-421c-bd81-4b2f160f0d71" (UID: "5037d4d9-6c79-421c-bd81-4b2f160f0d71"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:44:08.699376 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.699345 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5037d4d9-6c79-421c-bd81-4b2f160f0d71" (UID: "5037d4d9-6c79-421c-bd81-4b2f160f0d71"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:44:08.699376 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.699357 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5037d4d9-6c79-421c-bd81-4b2f160f0d71-kube-api-access-vsc7f" (OuterVolumeSpecName: "kube-api-access-vsc7f") pod "5037d4d9-6c79-421c-bd81-4b2f160f0d71" (UID: "5037d4d9-6c79-421c-bd81-4b2f160f0d71"). InnerVolumeSpecName "kube-api-access-vsc7f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:44:08.699542 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.699490 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5037d4d9-6c79-421c-bd81-4b2f160f0d71" (UID: "5037d4d9-6c79-421c-bd81-4b2f160f0d71"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:44:08.699667 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.699650 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-token" (OuterVolumeSpecName: "istio-token") pod "5037d4d9-6c79-421c-bd81-4b2f160f0d71" (UID: "5037d4d9-6c79-421c-bd81-4b2f160f0d71"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:44:08.745380 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.745342 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-lgpbn_26539bc2-ca5b-4dae-a116-5475c9ff79b5/manager/0.log" Jul 02 23:44:08.798169 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.798136 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsc7f\" (UniqueName: \"kubernetes.io/projected/5037d4d9-6c79-421c-bd81-4b2f160f0d71-kube-api-access-vsc7f\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:44:08.798169 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.798162 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-credential-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:44:08.798169 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.798174 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-data\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:44:08.798419 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.798184 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-workload-certs\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:44:08.798419 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.798193 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-token\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:44:08.798419 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.798200 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-workload-socket\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:44:08.798419 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.798208 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-envoy\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:44:08.798419 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.798215 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istiod-ca-cert\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:44:08.798419 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.798222 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5037d4d9-6c79-421c-bd81-4b2f160f0d71-istio-podinfo\") on node \"ip-10-0-136-243.ec2.internal\" DevicePath \"\"" Jul 02 23:44:08.962924 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:08.962825 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717250-kckqj_24e21038-26ea-4a86-a69c-45b52904e31b/cleanup/0.log" Jul 02 23:44:09.160556 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:09.160518 2583 generic.go:356] "Generic (PLEG): container finished" podID="5037d4d9-6c79-421c-bd81-4b2f160f0d71" containerID="88998b22e30ba7dfe5ffe01c57e105655000ebea868a4f4adf653813f2b76dc2" exitCode=0 Jul 02 23:44:09.160774 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:09.160573 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" Jul 02 23:44:09.160774 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:09.160615 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" event={"ID":"5037d4d9-6c79-421c-bd81-4b2f160f0d71","Type":"ContainerDied","Data":"88998b22e30ba7dfe5ffe01c57e105655000ebea868a4f4adf653813f2b76dc2"} Jul 02 23:44:09.160774 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:09.160684 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg" event={"ID":"5037d4d9-6c79-421c-bd81-4b2f160f0d71","Type":"ContainerDied","Data":"62418c5a455facde893da01d3d7bee72ed57c449e9c570864f491d1fa7ab6852"} Jul 02 23:44:09.160774 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:09.160717 2583 scope.go:117] "RemoveContainer" containerID="88998b22e30ba7dfe5ffe01c57e105655000ebea868a4f4adf653813f2b76dc2" Jul 02 23:44:09.170654 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:09.170635 2583 scope.go:117] "RemoveContainer" containerID="45f6b36c48dda0e75389fab6c7e9dc1a6c57559b22c3fe2a1a6612716a11c706" Jul 02 23:44:09.178743 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:09.178724 2583 scope.go:117] "RemoveContainer" containerID="88998b22e30ba7dfe5ffe01c57e105655000ebea868a4f4adf653813f2b76dc2" Jul 02 23:44:09.179277 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:44:09.179154 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88998b22e30ba7dfe5ffe01c57e105655000ebea868a4f4adf653813f2b76dc2\": container with ID starting with 88998b22e30ba7dfe5ffe01c57e105655000ebea868a4f4adf653813f2b76dc2 not found: ID does not exist" containerID="88998b22e30ba7dfe5ffe01c57e105655000ebea868a4f4adf653813f2b76dc2" Jul 02 23:44:09.179277 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:09.179185 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88998b22e30ba7dfe5ffe01c57e105655000ebea868a4f4adf653813f2b76dc2"} err="failed to get container status \"88998b22e30ba7dfe5ffe01c57e105655000ebea868a4f4adf653813f2b76dc2\": rpc error: code = NotFound desc = could not find container \"88998b22e30ba7dfe5ffe01c57e105655000ebea868a4f4adf653813f2b76dc2\": container with ID starting with 88998b22e30ba7dfe5ffe01c57e105655000ebea868a4f4adf653813f2b76dc2 not found: ID does not exist" Jul 02 23:44:09.179277 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:09.179203 2583 scope.go:117] "RemoveContainer" containerID="45f6b36c48dda0e75389fab6c7e9dc1a6c57559b22c3fe2a1a6612716a11c706" Jul 02 23:44:09.179569 ip-10-0-136-243 kubenswrapper[2583]: E0702 23:44:09.179510 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45f6b36c48dda0e75389fab6c7e9dc1a6c57559b22c3fe2a1a6612716a11c706\": container with ID starting with 45f6b36c48dda0e75389fab6c7e9dc1a6c57559b22c3fe2a1a6612716a11c706 not found: ID does not exist" containerID="45f6b36c48dda0e75389fab6c7e9dc1a6c57559b22c3fe2a1a6612716a11c706" Jul 02 23:44:09.179631 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:09.179542 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45f6b36c48dda0e75389fab6c7e9dc1a6c57559b22c3fe2a1a6612716a11c706"} err="failed to get container status \"45f6b36c48dda0e75389fab6c7e9dc1a6c57559b22c3fe2a1a6612716a11c706\": rpc error: code = NotFound desc = could not find container \"45f6b36c48dda0e75389fab6c7e9dc1a6c57559b22c3fe2a1a6612716a11c706\": container with ID starting with 45f6b36c48dda0e75389fab6c7e9dc1a6c57559b22c3fe2a1a6612716a11c706 not found: ID does not exist" Jul 02 23:44:09.181571 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:09.181549 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg"] Jul 02 23:44:09.185477 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:09.185457 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-f1cae1-openshift-default-649d5547c5-vcmrg"] Jul 02 23:44:09.185883 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:09.185866 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-n48md_c43b5a76-ece9-47b6-bab2-c44debab52e2/server/0.log" Jul 02 23:44:09.423036 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:09.423005 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fb7747bfc-bh6tc_7a14a32a-7506-4b68-b7b5-64dc31561af1/copy-manifests/0.log" Jul 02 23:44:09.429474 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:09.429437 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fb7747bfc-bh6tc_7a14a32a-7506-4b68-b7b5-64dc31561af1/manager/0.log" Jul 02 23:44:10.533094 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:10.533060 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5037d4d9-6c79-421c-bd81-4b2f160f0d71" path="/var/lib/kubelet/pods/5037d4d9-6c79-421c-bd81-4b2f160f0d71/volumes" Jul 02 23:44:11.037841 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:11.037808 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5b754c5d8d-vql6w_d2258cf2-7688-4cd2-ade2-01163ffd216a/authorino/0.log" Jul 02 23:44:11.154567 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:11.154536 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-dqv2s_592729d8-3f0f-401a-9bc4-6580f56faa57/manager/0.log" Jul 02 23:44:11.268183 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:11.268154 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-r9l69_3dab87d1-a71e-4de9-865c-42cd7ecf5392/manager/0.log" Jul 02 23:44:11.487876 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:11.487844 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-k5gq5_0e30f73b-2929-4898-818e-17000a744728/registry-server/0.log" Jul 02 23:44:11.838416 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:11.838386 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-pcqnn_0df8f401-e0e6-4bd6-b9a5-96f5e04c92cd/manager/0.log" Jul 02 23:44:12.178372 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:12.178289 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678m49s8_b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9/istio-proxy/0.log" Jul 02 23:44:12.421600 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:12.421568 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55fc97dffd-5wh2r_dfb1e9db-f7e1-4221-8dc8-41fd8f51f117/kube-auth-proxy/0.log" Jul 02 23:44:12.642294 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:12.642260 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-v8h54_35aad047-abe8-466e-9e01-ce87f1e2b080/istio-proxy/1.log" Jul 02 23:44:13.423618 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:13.423587 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-b88jb_079adb89-9b45-43b7-809b-256212373341/main/0.log" Jul 02 23:44:13.526142 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:13.526113 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-4g8gh_1d35a04f-f08f-417d-9c8b-6507e3f2f228/main/0.log" Jul 02 23:44:13.746137 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:13.746055 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-lqrqb_dfea2d44-f8af-4eba-bb7f-19da7387f04f/main/0.log" Jul 02 23:44:13.853467 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:13.853423 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-lr8z8_921fd0f0-6306-4f0d-b8d3-4279c8014f1f/main/0.log" Jul 02 23:44:20.972795 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:20.972751 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rs2g8_dcb0135c-18a2-4e25-9193-16f4e2a3b833/global-pull-secret-syncer/0.log" Jul 02 23:44:21.046615 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:21.046578 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ngwbb_a3a17f85-d07c-4017-935c-fb5dfe60dec9/konnectivity-agent/0.log" Jul 02 23:44:21.116129 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:21.116087 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-243.ec2.internal_20f20beb3812fd7a1ed1315ed6bb79ac/haproxy/0.log" Jul 02 23:44:26.108776 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:26.108740 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5b754c5d8d-vql6w_d2258cf2-7688-4cd2-ade2-01163ffd216a/authorino/0.log" Jul 02 23:44:26.133290 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:26.133262 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-dqv2s_592729d8-3f0f-401a-9bc4-6580f56faa57/manager/0.log" Jul 02 23:44:26.157872 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:26.157842 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-r9l69_3dab87d1-a71e-4de9-865c-42cd7ecf5392/manager/0.log" Jul 02 23:44:26.224490 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:26.224461 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-k5gq5_0e30f73b-2929-4898-818e-17000a744728/registry-server/0.log" Jul 02 23:44:26.428063 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:26.427970 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-pcqnn_0df8f401-e0e6-4bd6-b9a5-96f5e04c92cd/manager/0.log" Jul 02 23:44:28.293669 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:28.293639 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f5pkx_f2910e39-d10c-46c7-ac46-01534729dff7/node-exporter/0.log" Jul 02 23:44:28.320365 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:28.320337 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f5pkx_f2910e39-d10c-46c7-ac46-01534729dff7/kube-rbac-proxy/0.log" Jul 02 23:44:28.340822 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:28.340790 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f5pkx_f2910e39-d10c-46c7-ac46-01534729dff7/init-textfile/0.log" Jul 02 23:44:28.432490 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:28.432402 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-brftj_bd4a33d2-1122-4c89-8d4b-2c4b901e03b9/kube-rbac-proxy-main/0.log" Jul 02 23:44:28.453062 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:28.453028 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-brftj_bd4a33d2-1122-4c89-8d4b-2c4b901e03b9/kube-rbac-proxy-self/0.log" Jul 02 23:44:28.477515 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:28.477476 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-brftj_bd4a33d2-1122-4c89-8d4b-2c4b901e03b9/openshift-state-metrics/0.log" Jul 02 23:44:31.368826 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.368787 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr"] Jul 02 23:44:31.369255 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.369158 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5037d4d9-6c79-421c-bd81-4b2f160f0d71" containerName="istio-proxy" Jul 02 23:44:31.369255 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.369170 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5037d4d9-6c79-421c-bd81-4b2f160f0d71" containerName="istio-proxy" Jul 02 23:44:31.369255 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.369185 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="521c6dd9-fc83-412d-ac67-afadffde3488" containerName="istio-proxy" Jul 02 23:44:31.369255 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.369191 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="521c6dd9-fc83-412d-ac67-afadffde3488" containerName="istio-proxy" Jul 02 23:44:31.369255 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.369241 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="521c6dd9-fc83-412d-ac67-afadffde3488" containerName="istio-proxy" Jul 02 23:44:31.369255 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.369250 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="15ef0459-de5d-4547-93c8-679e0009e53a" containerName="istio-proxy" Jul 02 23:44:31.369255 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.369256 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5037d4d9-6c79-421c-bd81-4b2f160f0d71" containerName="istio-proxy" Jul 02 23:44:31.369508 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.369261 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="521c6dd9-fc83-412d-ac67-afadffde3488" containerName="istio-proxy" Jul 02 23:44:31.369508 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.369267 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5037d4d9-6c79-421c-bd81-4b2f160f0d71" containerName="istio-proxy" Jul 02 23:44:31.372300 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.372282 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" Jul 02 23:44:31.374302 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.374279 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rbq7r\"/\"kube-root-ca.crt\"" Jul 02 23:44:31.374495 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.374474 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rbq7r\"/\"default-dockercfg-jhnrh\"" Jul 02 23:44:31.375070 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.375051 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rbq7r\"/\"openshift-service-ca.crt\"" Jul 02 23:44:31.381592 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.381570 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr"] Jul 02 23:44:31.498582 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.498540 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d39f86b5-1e3c-4967-a7b1-1da2b860fcc6-lib-modules\") pod \"perf-node-gather-daemonset-xcbcr\" (UID: \"d39f86b5-1e3c-4967-a7b1-1da2b860fcc6\") " pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" Jul 02 23:44:31.498582 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.498585 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d39f86b5-1e3c-4967-a7b1-1da2b860fcc6-sys\") pod \"perf-node-gather-daemonset-xcbcr\" (UID: \"d39f86b5-1e3c-4967-a7b1-1da2b860fcc6\") " pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" Jul 02 23:44:31.498807 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.498611 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqclb\" (UniqueName: \"kubernetes.io/projected/d39f86b5-1e3c-4967-a7b1-1da2b860fcc6-kube-api-access-lqclb\") pod \"perf-node-gather-daemonset-xcbcr\" (UID: \"d39f86b5-1e3c-4967-a7b1-1da2b860fcc6\") " pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" Jul 02 23:44:31.498807 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.498666 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d39f86b5-1e3c-4967-a7b1-1da2b860fcc6-proc\") pod \"perf-node-gather-daemonset-xcbcr\" (UID: \"d39f86b5-1e3c-4967-a7b1-1da2b860fcc6\") " pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" Jul 02 23:44:31.498807 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.498722 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d39f86b5-1e3c-4967-a7b1-1da2b860fcc6-podres\") pod \"perf-node-gather-daemonset-xcbcr\" (UID: \"d39f86b5-1e3c-4967-a7b1-1da2b860fcc6\") " pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" Jul 02 23:44:31.600102 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.600044 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d39f86b5-1e3c-4967-a7b1-1da2b860fcc6-lib-modules\") pod \"perf-node-gather-daemonset-xcbcr\" (UID: \"d39f86b5-1e3c-4967-a7b1-1da2b860fcc6\") " pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" Jul 02 23:44:31.600254 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.600120 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d39f86b5-1e3c-4967-a7b1-1da2b860fcc6-sys\") pod \"perf-node-gather-daemonset-xcbcr\" (UID: \"d39f86b5-1e3c-4967-a7b1-1da2b860fcc6\") " pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" Jul 02 23:44:31.600254 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.600154 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqclb\" (UniqueName: \"kubernetes.io/projected/d39f86b5-1e3c-4967-a7b1-1da2b860fcc6-kube-api-access-lqclb\") pod \"perf-node-gather-daemonset-xcbcr\" (UID: \"d39f86b5-1e3c-4967-a7b1-1da2b860fcc6\") " pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" Jul 02 23:44:31.600254 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.600181 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d39f86b5-1e3c-4967-a7b1-1da2b860fcc6-proc\") pod \"perf-node-gather-daemonset-xcbcr\" (UID: \"d39f86b5-1e3c-4967-a7b1-1da2b860fcc6\") " pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" Jul 02 23:44:31.600254 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.600231 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d39f86b5-1e3c-4967-a7b1-1da2b860fcc6-podres\") pod \"perf-node-gather-daemonset-xcbcr\" (UID: \"d39f86b5-1e3c-4967-a7b1-1da2b860fcc6\") " pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" Jul 02 23:44:31.600416 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.600247 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d39f86b5-1e3c-4967-a7b1-1da2b860fcc6-lib-modules\") pod \"perf-node-gather-daemonset-xcbcr\" (UID: \"d39f86b5-1e3c-4967-a7b1-1da2b860fcc6\") " pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" Jul 02 23:44:31.600416 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.600245 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d39f86b5-1e3c-4967-a7b1-1da2b860fcc6-sys\") pod \"perf-node-gather-daemonset-xcbcr\" (UID: \"d39f86b5-1e3c-4967-a7b1-1da2b860fcc6\") " pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" Jul 02 23:44:31.600416 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.600280 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d39f86b5-1e3c-4967-a7b1-1da2b860fcc6-proc\") pod \"perf-node-gather-daemonset-xcbcr\" (UID: \"d39f86b5-1e3c-4967-a7b1-1da2b860fcc6\") " pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" Jul 02 23:44:31.600416 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.600340 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d39f86b5-1e3c-4967-a7b1-1da2b860fcc6-podres\") pod \"perf-node-gather-daemonset-xcbcr\" (UID: \"d39f86b5-1e3c-4967-a7b1-1da2b860fcc6\") " pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" Jul 02 23:44:31.615136 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.615102 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqclb\" (UniqueName: \"kubernetes.io/projected/d39f86b5-1e3c-4967-a7b1-1da2b860fcc6-kube-api-access-lqclb\") pod \"perf-node-gather-daemonset-xcbcr\" (UID: \"d39f86b5-1e3c-4967-a7b1-1da2b860fcc6\") " pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" Jul 02 23:44:31.683388 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.683306 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" Jul 02 23:44:31.817707 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:31.817674 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr"] Jul 02 23:44:31.821549 ip-10-0-136-243 kubenswrapper[2583]: W0702 23:44:31.821519 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd39f86b5_1e3c_4967_a7b1_1da2b860fcc6.slice/crio-9541e4472b9c35c5507dabd8749569a854b51463bf80827e7b2a660e51f46e47 WatchSource:0}: Error finding container 9541e4472b9c35c5507dabd8749569a854b51463bf80827e7b2a660e51f46e47: Status 404 returned error can't find the container with id 9541e4472b9c35c5507dabd8749569a854b51463bf80827e7b2a660e51f46e47 Jul 02 23:44:32.270182 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:32.270150 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" event={"ID":"d39f86b5-1e3c-4967-a7b1-1da2b860fcc6","Type":"ContainerStarted","Data":"910fd197c1f7294ac9514b1b659f80bf13c85d03347747c5ac4820f862a9dd59"} Jul 02 23:44:32.270365 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:32.270199 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" event={"ID":"d39f86b5-1e3c-4967-a7b1-1da2b860fcc6","Type":"ContainerStarted","Data":"9541e4472b9c35c5507dabd8749569a854b51463bf80827e7b2a660e51f46e47"} Jul 02 23:44:32.270365 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:32.270292 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" Jul 02 23:44:32.284451 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:32.284404 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" podStartSLOduration=1.284391012 podStartE2EDuration="1.284391012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:44:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:44:32.283040545 +0000 UTC m=+2236.305858658" watchObservedRunningTime="2026-07-02 23:44:32.284391012 +0000 UTC m=+2236.307209124" Jul 02 23:44:33.104340 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:33.104294 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2vvfs_fb8c01ab-cb1f-403b-aef4-c4a7143b7c79/dns/0.log" Jul 02 23:44:33.122755 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:33.122711 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2vvfs_fb8c01ab-cb1f-403b-aef4-c4a7143b7c79/kube-rbac-proxy/0.log" Jul 02 23:44:33.228033 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:33.228006 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b2nvt_ad87676b-0e54-4723-895d-9e19afedfca7/dns-node-resolver/0.log" Jul 02 23:44:33.790092 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:33.790065 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bv9xn_2055d544-3273-4518-b065-e41ceba96443/node-ca/0.log" Jul 02 23:44:34.688810 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:34.688770 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678m49s8_b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9/istio-proxy/0.log" Jul 02 23:44:34.833416 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:34.833388 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55fc97dffd-5wh2r_dfb1e9db-f7e1-4221-8dc8-41fd8f51f117/kube-auth-proxy/0.log" Jul 02 23:44:34.884380 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:34.884349 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-v8h54_35aad047-abe8-466e-9e01-ce87f1e2b080/istio-proxy/0.log" Jul 02 23:44:34.913969 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:34.913937 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-v8h54_35aad047-abe8-466e-9e01-ce87f1e2b080/istio-proxy/1.log" Jul 02 23:44:35.728904 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:35.728873 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vblxw_952de78d-e0fb-4c00-88d6-ca117114fd67/serve-healthcheck-canary/0.log" Jul 02 23:44:36.325175 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:36.325149 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bqx8m_57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02/kube-rbac-proxy/0.log" Jul 02 23:44:36.344975 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:36.344948 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bqx8m_57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02/exporter/0.log" Jul 02 23:44:36.367017 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:36.366991 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bqx8m_57a1c6c4-2a20-45c5-b9e8-fd77c9a22b02/extractor/0.log" Jul 02 23:44:38.286377 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:38.286351 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rbq7r/perf-node-gather-daemonset-xcbcr" Jul 02 23:44:38.440583 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:38.440550 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-5nccz_3dc2f7e3-a8b3-4280-a2f1-828b24282c70/manager/0.log" Jul 02 23:44:38.546805 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:38.546766 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-lgpbn_26539bc2-ca5b-4dae-a116-5475c9ff79b5/manager/0.log" Jul 02 23:44:38.602290 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:38.602251 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717250-kckqj_24e21038-26ea-4a86-a69c-45b52904e31b/cleanup/0.log" Jul 02 23:44:38.701596 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:38.701564 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-n48md_c43b5a76-ece9-47b6-bab2-c44debab52e2/server/0.log" Jul 02 23:44:38.795376 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:38.795342 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fb7747bfc-bh6tc_7a14a32a-7506-4b68-b7b5-64dc31561af1/manager/0.log" Jul 02 23:44:38.814142 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:38.814055 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fb7747bfc-bh6tc_7a14a32a-7506-4b68-b7b5-64dc31561af1/copy-manifests/0.log" Jul 02 23:44:40.359611 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:40.359577 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-rbcgl_7f1b26aa-e4ea-47ce-b953-0ecef03d1b74/openshift-lws-operator/0.log" Jul 02 23:44:41.487765 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:41.487720 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678m49s8_b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9/istio-proxy/0.log" Jul 02 23:44:41.605468 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:41.605434 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55fc97dffd-5wh2r_dfb1e9db-f7e1-4221-8dc8-41fd8f51f117/kube-auth-proxy/0.log" Jul 02 23:44:41.640790 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:41.640743 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-v8h54_35aad047-abe8-466e-9e01-ce87f1e2b080/istio-proxy/0.log" Jul 02 23:44:41.668944 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:41.668881 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-v8h54_35aad047-abe8-466e-9e01-ce87f1e2b080/istio-proxy/1.log" Jul 02 23:44:42.959944 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:42.959918 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678m49s8_b5cfec4f-0a2b-46e4-9e41-3545eba0b2d9/istio-proxy/0.log" Jul 02 23:44:43.089686 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:43.089655 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55fc97dffd-5wh2r_dfb1e9db-f7e1-4221-8dc8-41fd8f51f117/kube-auth-proxy/0.log" Jul 02 23:44:43.126832 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:43.126797 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-v8h54_35aad047-abe8-466e-9e01-ce87f1e2b080/istio-proxy/0.log" Jul 02 23:44:43.153554 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:43.153519 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-v8h54_35aad047-abe8-466e-9e01-ce87f1e2b080/istio-proxy/1.log" Jul 02 23:44:47.090640 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:47.090612 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xt994_511288f9-603d-46d3-aaf1-f1725e8f3226/kube-multus-additional-cni-plugins/0.log" Jul 02 23:44:47.109260 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:47.109234 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xt994_511288f9-603d-46d3-aaf1-f1725e8f3226/egress-router-binary-copy/0.log" Jul 02 23:44:47.127109 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:47.127083 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xt994_511288f9-603d-46d3-aaf1-f1725e8f3226/cni-plugins/0.log" Jul 02 23:44:47.145500 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:47.145472 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xt994_511288f9-603d-46d3-aaf1-f1725e8f3226/bond-cni-plugin/0.log" Jul 02 23:44:47.163525 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:47.163502 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xt994_511288f9-603d-46d3-aaf1-f1725e8f3226/routeoverride-cni/0.log" Jul 02 23:44:47.181615 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:47.181589 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xt994_511288f9-603d-46d3-aaf1-f1725e8f3226/whereabouts-cni-bincopy/0.log" Jul 02 23:44:47.200373 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:47.200340 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xt994_511288f9-603d-46d3-aaf1-f1725e8f3226/whereabouts-cni/0.log" Jul 02 23:44:47.230434 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:47.230402 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-cnq78_d952dcf9-8a86-487d-9cfb-b6c592a17982/kube-multus/0.log" Jul 02 23:44:47.360015 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:47.359978 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xqwp5_e3f048d9-2670-488f-9a21-73f967c069b7/network-metrics-daemon/0.log" Jul 02 23:44:47.377050 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:47.377021 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xqwp5_e3f048d9-2670-488f-9a21-73f967c069b7/kube-rbac-proxy/0.log" Jul 02 23:44:48.136655 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:48.136622 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcv6k_cb571463-b959-4b84-bba1-2b22b22157fd/ovn-controller/0.log" Jul 02 23:44:48.150968 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:48.150943 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcv6k_cb571463-b959-4b84-bba1-2b22b22157fd/ovn-acl-logging/0.log" Jul 02 23:44:48.174454 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:48.174406 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcv6k_cb571463-b959-4b84-bba1-2b22b22157fd/kube-rbac-proxy-node/0.log" Jul 02 23:44:48.196687 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:48.196656 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcv6k_cb571463-b959-4b84-bba1-2b22b22157fd/kube-rbac-proxy-ovn-metrics/0.log" Jul 02 23:44:48.215057 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:48.215030 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcv6k_cb571463-b959-4b84-bba1-2b22b22157fd/northd/0.log" Jul 02 23:44:48.232659 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:48.232635 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcv6k_cb571463-b959-4b84-bba1-2b22b22157fd/nbdb/0.log" Jul 02 23:44:48.250085 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:48.250059 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcv6k_cb571463-b959-4b84-bba1-2b22b22157fd/sbdb/0.log" Jul 02 23:44:48.300603 ip-10-0-136-243 kubenswrapper[2583]: I0702 23:44:48.300569 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcv6k_cb571463-b959-4b84-bba1-2b22b22157fd/ovnkube-controller/0.log"