Jul 08 23:00:55.411949 ip-10-0-135-106 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 23:00:55.411963 ip-10-0-135-106 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 23:00:55.411973 ip-10-0-135-106 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 23:00:55.412322 ip-10-0-135-106 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 23:01:05.592090 ip-10-0-135-106 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 23:01:05.592109 ip-10-0-135-106 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 8a7170d0143a45ce8b2035b87568a52a -- Jul 08 23:02:58.120925 ip-10-0-135-106 systemd[1]: Starting Kubernetes Kubelet... Jul 08 23:02:58.575824 ip-10-0-135-106 kubenswrapper[2548]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 23:02:58.575824 ip-10-0-135-106 kubenswrapper[2548]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 23:02:58.575824 ip-10-0-135-106 kubenswrapper[2548]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 23:02:58.575824 ip-10-0-135-106 kubenswrapper[2548]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 23:02:58.575824 ip-10-0-135-106 kubenswrapper[2548]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 23:02:58.575824 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.563018 2548 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 23:02:58.575824 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570650 2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 23:02:58.575824 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570669 2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 23:02:58.575824 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570674 2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 23:02:58.575824 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570677 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 23:02:58.575824 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570680 2548 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 23:02:58.575824 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570684 2548 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 23:02:58.575824 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570687 2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 23:02:58.575824 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570691 2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570694 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570697 2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570703 2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570708 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570712 2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570715 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570718 2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570721 2548 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570724 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570727 2548 feature_gate.go:328] unrecognized feature gate: Example Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570730 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570733 2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570735 2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570738 2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570741 2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570745 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570748 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570751 2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570754 2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 23:02:58.577084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570757 2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 23:02:58.577699 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570759 2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 23:02:58.577699 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570762 2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 23:02:58.577699 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570765 2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 23:02:58.577699 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570768 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 23:02:58.577699 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570779 2548 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 23:02:58.577699 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570782 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 23:02:58.577699 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570785 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 23:02:58.577699 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570788 2548 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 23:02:58.577699 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570792 2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 23:02:58.577699 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570797 2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 23:02:58.577699 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570800 2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 23:02:58.577699 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570804 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 23:02:58.577699 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570807 2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 23:02:58.577699 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570810 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 23:02:58.577699 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570813 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 23:02:58.577699 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570816 2548 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 23:02:58.577699 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570819 2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 23:02:58.577699 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570822 2548 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 23:02:58.577699 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570826 2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570829 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570832 2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570835 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570837 2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570840 2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570843 2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570845 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570848 2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570853 2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570856 2548 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570859 2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570862 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570865 2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570868 2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570871 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570874 2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570877 2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570880 2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570883 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 23:02:58.578429 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570886 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 23:02:58.579022 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570889 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 23:02:58.579022 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570891 2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 23:02:58.579022 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570894 2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 23:02:58.579022 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570897 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 23:02:58.579022 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570901 2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 23:02:58.579022 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570904 2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 23:02:58.579022 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570906 2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 23:02:58.579022 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570909 2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 23:02:58.579022 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570912 2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 23:02:58.579022 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570915 2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 23:02:58.579022 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570917 2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 23:02:58.579022 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570920 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 23:02:58.579022 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570923 2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 23:02:58.579022 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570929 2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 23:02:58.579022 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570932 2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 23:02:58.579022 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570935 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 23:02:58.579022 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570939 2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 23:02:58.579022 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570942 2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 23:02:58.579022 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570944 2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570947 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570950 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570952 2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570956 2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570959 2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570962 2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570965 2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570968 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570971 2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570974 2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570977 2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570979 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.570982 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571816 2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571826 2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571830 2548 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571834 2548 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571837 2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571840 2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 23:02:58.579755 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571843 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571846 2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571849 2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571852 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571854 2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571857 2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571860 2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571863 2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571865 2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571868 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571871 2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571874 2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571877 2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571880 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571883 2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571886 2548 feature_gate.go:328] unrecognized feature gate: Example Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571890 2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571892 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571895 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571898 2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 23:02:58.580411 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571901 2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571904 2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571907 2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571909 2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571912 2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571915 2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571918 2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571920 2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571924 2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571927 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571929 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571932 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571935 2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571937 2548 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571940 2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571942 2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571945 2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571948 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571951 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571954 2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 23:02:58.581342 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571956 2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571959 2548 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571961 2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571964 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571967 2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571969 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571972 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571975 2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571978 2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571981 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571984 2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571987 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571990 2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571993 2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.571997 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572000 2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572003 2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572005 2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572008 2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572011 2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 23:02:58.581954 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572013 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 23:02:58.582803 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572016 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 23:02:58.582803 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572019 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 23:02:58.582803 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572022 2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 23:02:58.582803 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572024 2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 23:02:58.582803 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572027 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 23:02:58.582803 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572030 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 23:02:58.582803 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572032 2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 23:02:58.582803 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572035 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 23:02:58.582803 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572040 2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 23:02:58.582803 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572044 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 23:02:58.582803 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572047 2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 23:02:58.582803 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572050 2548 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 23:02:58.582803 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572053 2548 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 23:02:58.582803 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572057 2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 23:02:58.582803 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572061 2548 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 23:02:58.582803 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572064 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 23:02:58.582803 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572068 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 23:02:58.582803 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572072 2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 23:02:58.582803 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572076 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572079 2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572082 2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572089 2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572091 2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572095 2548 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572098 2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572101 2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572103 2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572106 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572109 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572112 2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572114 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572117 2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572520 2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572525 2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572528 2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572531 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572534 2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572537 2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 23:02:58.583368 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572540 2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 23:02:58.714089 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572543 2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 23:02:58.714089 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572546 2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 23:02:58.714089 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572549 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 23:02:58.714089 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572551 2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 23:02:58.714089 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572554 2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 23:02:58.714089 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572557 2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 23:02:58.714089 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572559 2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 23:02:58.714089 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572562 2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 23:02:58.714089 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572565 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 23:02:58.714089 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572568 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 23:02:58.714089 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572571 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 23:02:58.714089 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572574 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 23:02:58.714089 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572577 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 23:02:58.714089 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572580 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 23:02:58.714089 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572585 2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 23:02:58.714089 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572587 2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 23:02:58.714089 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572590 2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 23:02:58.714089 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572593 2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 23:02:58.714089 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572595 2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 23:02:58.647693 ip-10-0-135-106 systemd[1]: Started Kubernetes Kubelet. Jul 08 23:02:58.714936 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572598 2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 23:02:58.714936 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572601 2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 23:02:58.714936 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572603 2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 23:02:58.714936 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572606 2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 23:02:58.714936 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572609 2548 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 23:02:58.714936 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572611 2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 23:02:58.714936 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572614 2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 23:02:58.714936 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572617 2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 23:02:58.714936 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572619 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 23:02:58.714936 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572623 2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 23:02:58.714936 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572627 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 23:02:58.714936 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572630 2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 23:02:58.714936 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572633 2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 23:02:58.714936 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572636 2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 23:02:58.714936 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572638 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 23:02:58.714936 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572641 2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 23:02:58.714936 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572645 2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 23:02:58.714936 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572649 2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 23:02:58.714936 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572652 2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 23:02:58.716007 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572655 2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 23:02:58.716007 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572661 2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 23:02:58.716007 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572666 2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 23:02:58.716007 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572685 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 23:02:58.716007 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572689 2548 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 23:02:58.716007 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572692 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 23:02:58.716007 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572695 2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 23:02:58.716007 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572698 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 23:02:58.716007 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572702 2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 23:02:58.716007 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572710 2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 23:02:58.716007 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572713 2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 23:02:58.716007 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572716 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 23:02:58.716007 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572718 2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 23:02:58.716007 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572721 2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 23:02:58.716007 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572724 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 23:02:58.716007 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572727 2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 23:02:58.716007 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572730 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 23:02:58.716007 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572733 2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 23:02:58.716007 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572735 2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572738 2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572741 2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572744 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572747 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572749 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572752 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572757 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572759 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572762 2548 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572765 2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572767 2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572770 2548 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572773 2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572776 2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572780 2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572783 2548 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572786 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572789 2548 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572792 2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572795 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 23:02:58.716827 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572798 2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572803 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572805 2548 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572809 2548 feature_gate.go:328] unrecognized feature gate: Example Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572811 2548 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572814 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572817 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572820 2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572822 2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572825 2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572828 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572830 2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572833 2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572836 2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.572838 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.572914 2548 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.572921 2548 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.572928 2548 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.572935 2548 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.572942 2548 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.572947 2548 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 23:02:58.717966 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.572954 2548 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.572961 2548 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.572966 2548 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.572969 2548 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.572973 2548 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.572979 2548 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.572982 2548 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.572986 2548 flags.go:64] FLAG: --cgroup-root="" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.572989 2548 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.572992 2548 flags.go:64] FLAG: --client-ca-file="" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.572996 2548 flags.go:64] FLAG: --cloud-provider="external" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573000 2548 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573007 2548 flags.go:64] FLAG: --cluster-domain="" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573010 2548 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573013 2548 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573016 2548 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573020 2548 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573024 2548 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573027 2548 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573031 2548 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573034 2548 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573037 2548 flags.go:64] FLAG: --contention-profiling="false" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573041 2548 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573044 2548 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 23:02:58.718773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573047 2548 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573050 2548 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573055 2548 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573058 2548 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573061 2548 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573064 2548 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573067 2548 flags.go:64] FLAG: --enable-server="true" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573070 2548 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573076 2548 flags.go:64] FLAG: --event-burst="100" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573079 2548 flags.go:64] FLAG: --event-qps="50" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573082 2548 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573085 2548 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573089 2548 flags.go:64] FLAG: --eviction-hard="" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573094 2548 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573097 2548 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573101 2548 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573104 2548 flags.go:64] FLAG: --eviction-soft="" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573107 2548 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573110 2548 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573114 2548 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573119 2548 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573122 2548 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573125 2548 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573128 2548 flags.go:64] FLAG: --feature-gates="" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573132 2548 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 23:02:58.719974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573135 2548 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573139 2548 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573142 2548 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573145 2548 flags.go:64] FLAG: --healthz-port="10248" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573148 2548 flags.go:64] FLAG: --help="false" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573151 2548 flags.go:64] FLAG: --hostname-override="ip-10-0-135-106.ec2.internal" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573155 2548 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573158 2548 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573161 2548 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573165 2548 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573168 2548 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573171 2548 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573174 2548 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573177 2548 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573180 2548 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573183 2548 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573186 2548 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573189 2548 flags.go:64] FLAG: --kube-reserved="" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573192 2548 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573195 2548 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573200 2548 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573203 2548 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573206 2548 flags.go:64] FLAG: --lock-file="" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573209 2548 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 23:02:58.720989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573212 2548 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573215 2548 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573220 2548 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573225 2548 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573228 2548 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573231 2548 flags.go:64] FLAG: --logging-format="text" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573235 2548 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573238 2548 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573241 2548 flags.go:64] FLAG: --manifest-url="" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573244 2548 flags.go:64] FLAG: --manifest-url-header="" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573249 2548 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573252 2548 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573256 2548 flags.go:64] FLAG: --max-pods="110" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573259 2548 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573262 2548 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573265 2548 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573269 2548 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573272 2548 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573275 2548 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573278 2548 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573285 2548 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573289 2548 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573292 2548 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 23:02:58.722061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573295 2548 flags.go:64] FLAG: --pod-cidr="" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573298 2548 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573304 2548 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573307 2548 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573311 2548 flags.go:64] FLAG: --pods-per-core="0" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573314 2548 flags.go:64] FLAG: --port="10250" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573318 2548 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573321 2548 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-004a1a574ce61d796" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573325 2548 flags.go:64] FLAG: --qos-reserved="" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573328 2548 flags.go:64] FLAG: --read-only-port="10255" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573331 2548 flags.go:64] FLAG: --register-node="true" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573334 2548 flags.go:64] FLAG: --register-with-taints="" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573339 2548 flags.go:64] FLAG: --registry-burst="10" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573343 2548 flags.go:64] FLAG: --registry-qps="5" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573346 2548 flags.go:64] FLAG: --reserved-cpus="" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573349 2548 flags.go:64] FLAG: --reserved-memory="" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573353 2548 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573356 2548 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573360 2548 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573363 2548 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573366 2548 flags.go:64] FLAG: --runonce="false" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573368 2548 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573371 2548 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573375 2548 flags.go:64] FLAG: --seccomp-default="false" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573378 2548 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 23:02:58.722823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573381 2548 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573384 2548 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573387 2548 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573390 2548 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573393 2548 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573397 2548 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573399 2548 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573402 2548 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573406 2548 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573409 2548 flags.go:64] FLAG: --system-cgroups="" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573412 2548 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573418 2548 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573421 2548 flags.go:64] FLAG: --tls-cert-file="" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573426 2548 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573430 2548 flags.go:64] FLAG: --tls-min-version="" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573433 2548 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573436 2548 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573439 2548 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573442 2548 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573446 2548 flags.go:64] FLAG: --v="2" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573450 2548 flags.go:64] FLAG: --version="false" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573456 2548 flags.go:64] FLAG: --vmodule="" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573465 2548 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.573469 2548 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573591 2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 23:02:58.723756 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573596 2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 23:02:58.724568 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573599 2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 23:02:58.724568 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573602 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 23:02:58.724568 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573604 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 23:02:58.724568 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573607 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 23:02:58.724568 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573610 2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 23:02:58.724568 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573612 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 23:02:58.724568 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573615 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 23:02:58.724568 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573618 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 23:02:58.724568 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573620 2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 23:02:58.724568 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573624 2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 23:02:58.724568 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573626 2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 23:02:58.724568 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573629 2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 23:02:58.724568 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573632 2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 23:02:58.724568 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573634 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 23:02:58.724568 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573637 2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 23:02:58.724568 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573639 2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 23:02:58.724568 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573642 2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 23:02:58.724568 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573645 2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 23:02:58.724568 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573647 2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 23:02:58.980084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573651 2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 23:02:58.980084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573654 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 23:02:58.980084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573657 2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 23:02:58.980084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573660 2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 23:02:58.980084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573663 2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 23:02:58.980084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573666 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 23:02:58.980084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573684 2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 23:02:58.980084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573687 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 23:02:58.980084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573690 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 23:02:58.980084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573693 2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 23:02:58.980084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573696 2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 23:02:58.980084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573699 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 23:02:58.980084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573702 2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 23:02:58.980084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573705 2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 23:02:58.980084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573707 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 23:02:58.980084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573710 2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 23:02:58.980084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573713 2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 23:02:58.980084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573716 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 23:02:58.980084 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573719 2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573723 2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573727 2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573730 2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573733 2548 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573736 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573739 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573742 2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573744 2548 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573748 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573750 2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573753 2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573755 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573758 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573763 2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573766 2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573769 2548 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573771 2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573774 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573777 2548 feature_gate.go:328] unrecognized feature gate: Example Jul 08 23:02:58.982185 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573781 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573784 2548 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573786 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573789 2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573792 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573795 2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573798 2548 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573801 2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573804 2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573806 2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573809 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573812 2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573814 2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573817 2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573820 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573823 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573826 2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573828 2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573831 2548 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573834 2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 23:02:58.985794 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573837 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 23:02:58.986900 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573841 2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 23:02:58.986900 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573844 2548 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 23:02:58.986900 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573848 2548 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 23:02:58.986900 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573850 2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 23:02:58.986900 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573853 2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 23:02:58.986900 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573858 2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 23:02:58.986900 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573861 2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 23:02:58.986900 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573864 2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 23:02:58.986900 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573868 2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 23:02:58.986900 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573872 2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 23:02:58.986900 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573875 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 23:02:58.986900 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573880 2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 23:02:58.986900 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573883 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 23:02:58.986900 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573886 2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 23:02:58.986900 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573889 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 23:02:58.986900 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573893 2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 23:02:58.986900 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573896 2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 23:02:58.986900 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573899 2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 23:02:58.986900 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.573902 2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 23:02:58.987847 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.574501 2548 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 23:02:58.987847 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.581650 2548 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 23:02:58.987847 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.581710 2548 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 23:02:58.987847 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.582770 2548 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 23:02:58.987847 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.582779 2548 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 23:02:58.987847 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582844 2548 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 23:02:58.987847 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582851 2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 23:02:58.987847 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582855 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 23:02:58.987847 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582858 2548 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 23:02:58.987847 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582862 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 23:02:58.987847 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582866 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 23:02:58.987847 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582869 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 23:02:58.987847 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582872 2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 23:02:58.987847 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582875 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 23:02:58.987847 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582878 2548 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582881 2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582883 2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582886 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582889 2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582892 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582895 2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582898 2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582901 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582904 2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582907 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582910 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582913 2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582916 2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582919 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582922 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582925 2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582927 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582930 2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582933 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 23:02:58.991306 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582937 2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 23:02:58.992481 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582940 2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 23:02:58.992481 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582942 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 23:02:58.992481 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582945 2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 23:02:58.992481 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582948 2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 23:02:58.992481 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582951 2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 23:02:58.992481 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582953 2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 23:02:58.992481 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582958 2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 23:02:58.992481 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582961 2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 23:02:58.992481 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582964 2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 23:02:58.992481 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582967 2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 23:02:58.992481 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582971 2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 23:02:58.992481 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582975 2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 23:02:58.992481 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582978 2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 23:02:58.992481 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582981 2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 23:02:58.992481 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582983 2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 23:02:58.992481 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582987 2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 23:02:58.992481 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582990 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 23:02:58.992481 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582993 2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 23:02:58.992481 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.582996 2548 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583000 2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583003 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583006 2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583009 2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583013 2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583017 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583020 2548 feature_gate.go:328] unrecognized feature gate: Example Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583022 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583025 2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583028 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583031 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583033 2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583036 2548 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583040 2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583043 2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583045 2548 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583048 2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583052 2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583054 2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 23:02:58.993410 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583057 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583060 2548 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583062 2548 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583065 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583068 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583070 2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583073 2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583076 2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583078 2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583081 2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583084 2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583087 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583089 2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583092 2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583095 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583098 2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583101 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583104 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583107 2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583110 2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 23:02:58.995508 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583113 2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 23:02:58.996714 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583115 2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 23:02:58.996714 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583118 2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 23:02:58.996714 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583121 2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 23:02:58.996714 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583124 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 23:02:58.996714 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583127 2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 23:02:58.996714 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583130 2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 23:02:58.996714 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583133 2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 23:02:58.996714 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583136 2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 23:02:58.996714 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583139 2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 23:02:58.996714 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583142 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 23:02:58.996714 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.583147 2548 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 23:02:58.996714 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583270 2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 23:02:58.996714 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583275 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 23:02:58.996714 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583279 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 23:02:58.996714 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583281 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 23:02:58.999477 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583284 2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 23:02:58.999477 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583287 2548 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 23:02:58.999477 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583290 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 23:02:58.999477 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583293 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 23:02:58.999477 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583296 2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 23:02:58.999477 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583299 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 23:02:58.999477 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583302 2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 23:02:58.999477 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583304 2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 23:02:58.999477 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583307 2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 23:02:58.999477 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583310 2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 23:02:58.999477 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583314 2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 23:02:58.999477 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583317 2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 23:02:58.999477 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583321 2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 23:02:58.999477 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583324 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 23:02:58.999477 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583327 2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 23:02:58.999477 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583330 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 23:02:58.999477 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583334 2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 23:02:58.999477 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583336 2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 23:02:58.999477 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583339 2548 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 23:02:59.000416 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583343 2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 23:02:59.000416 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583347 2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 23:02:59.000416 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583350 2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 23:02:59.000416 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583353 2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 23:02:59.000416 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583356 2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 23:02:59.000416 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583359 2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 23:02:59.000416 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583362 2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 23:02:59.000416 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583365 2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 23:02:59.000416 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583368 2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 23:02:59.000416 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583370 2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 23:02:59.000416 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583374 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 23:02:59.000416 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583376 2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 23:02:59.000416 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583379 2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 23:02:59.000416 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583382 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 23:02:59.000416 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583385 2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 23:02:59.000416 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583388 2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 23:02:59.000416 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583391 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 23:02:59.000416 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583393 2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 23:02:59.000416 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583396 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583398 2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583401 2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583404 2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583407 2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583410 2548 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583413 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583415 2548 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583418 2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583421 2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583424 2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583427 2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583430 2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583432 2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583435 2548 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583438 2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583440 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583443 2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583446 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583449 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 23:02:59.199387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583452 2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583455 2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583457 2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583460 2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583463 2548 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583465 2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583468 2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583471 2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583473 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583476 2548 feature_gate.go:328] unrecognized feature gate: Example Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583479 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583481 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583484 2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583487 2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583490 2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583492 2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583496 2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583500 2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583503 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583506 2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 23:02:59.200065 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583509 2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 23:02:59.201406 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583513 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 23:02:59.201406 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583516 2548 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 23:02:59.201406 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583519 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 23:02:59.201406 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583522 2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 23:02:59.201406 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583525 2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 23:02:59.201406 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583527 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 23:02:59.201406 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583530 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 23:02:59.201406 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583533 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 23:02:59.201406 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583536 2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 23:02:59.201406 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583539 2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 23:02:59.201406 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583541 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 23:02:59.201406 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583544 2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 23:02:59.201406 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583547 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 23:02:59.201406 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583550 2548 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 23:02:59.201406 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583553 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 23:02:59.201406 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583556 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 23:02:59.201406 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:58.583558 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 23:02:59.201959 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.583564 2548 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 23:02:59.201959 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.583711 2548 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 23:02:59.201959 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.585908 2548 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 23:02:59.201959 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.587434 2548 server.go:1019] "Starting client certificate rotation" Jul 08 23:02:59.201959 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.587540 2548 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 23:02:59.201959 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.587570 2548 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 23:02:59.201959 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.590462 2548 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 23:02:59.201959 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.594921 2548 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 23:02:59.201959 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.605013 2548 log.go:25] "Validated CRI v1 runtime API" Jul 08 23:02:59.201959 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.607709 2548 log.go:25] "Validated CRI v1 image API" Jul 08 23:02:59.201959 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.608801 2548 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 23:02:59.201959 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.613227 2548 fs.go:135] Filesystem UUIDs: map[3a2f0100-6227-4fce-bf0c-df6bda0e6ffb:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8f33dcca-d26f-46ab-aa62-b5692d06ceae:/dev/nvme0n1p3] Jul 08 23:02:59.202345 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.613245 2548 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 23:02:59.413749 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.618700 2548 manager.go:216] Machine: {Timestamp:2026-07-08 23:02:58.617225966 +0000 UTC m=+0.377721013 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3031943 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28c0351161efab645db5ad83e25d2f SystemUUID:ec28c035-1161-efab-645d-b5ad83e25d2f BootID:8a7170d0-143a-45ce-8b20-35b87568a52a Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:22:1d:3c:6d:8f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:22:1d:3c:6d:8f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5e:60:10:b7:10:1c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 23:02:59.413749 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.618814 2548 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 23:02:59.413749 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.618906 2548 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 23:02:59.413749 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.619474 2548 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 23:02:59.413749 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.622180 2548 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 23:02:59.413749 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.622211 2548 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-106.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 23:02:59.413749 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.623017 2548 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.623027 2548 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.623036 2548 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.623593 2548 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.624804 2548 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.627579 2548 state_mem.go:36] "Initialized new in-memory state store" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.627707 2548 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.627737 2548 kubelet.go:486] "Attempting to sync node with API server" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.627748 2548 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.627759 2548 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.627769 2548 kubelet.go:398] "Adding apiserver pod source" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.627778 2548 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.628110 2548 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.628125 2548 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.630328 2548 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.632236 2548 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.632275 2548 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.640061 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.640110 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.640127 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.640140 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.640154 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.640178 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.640195 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.640209 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.640223 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.640271 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.640289 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.640299 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.641149 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.641167 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.642461 2548 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z2pqc" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.646573 2548 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z2pqc" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.646588 2548 server.go:1268] "Started kubelet" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.646828 2548 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.646908 2548 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.646967 2548 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.649407 2548 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.649601 2548 server.go:310] "Adding debug handlers to kubelet server" Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.651621 2548 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-106.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 23:02:59.415309 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:58.652118 2548 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:58.652185 2548 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-106.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:58.653703 2548 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.655661 2548 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.655691 2548 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:58.655805 2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-106.ec2.internal\" not found" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.656431 2548 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.656434 2548 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.656450 2548 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.656549 2548 reconstruct.go:97] "Volume reconstruction finished" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.656560 2548 reconciler.go:29] "Reconciler: start to sync state" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.656669 2548 factory.go:55] Registering systemd factory Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.656687 2548 factory.go:223] Registration of the systemd container factory successfully Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.656961 2548 factory.go:153] Registering CRI-O factory Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.656972 2548 factory.go:223] Registration of the crio container factory successfully Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.657027 2548 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.657052 2548 factory.go:103] Registering Raw factory Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.657067 2548 manager.go:1196] Started watching for new ooms in manager Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.657908 2548 manager.go:349] Starting recovery of all containers Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.680294 2548 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:58.684694 2548 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-106.ec2.internal\" not found" node="ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.686731 2548 manager.go:354] Recovery completed Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.691796 2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.693932 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasSufficientMemory" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.693966 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.693981 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasSufficientPID" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.694509 2548 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.694515 2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.694519 2548 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.694562 2548 state_mem.go:36] "Initialized new in-memory state store" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.695288 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasSufficientMemory" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.695323 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.695344 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasSufficientPID" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.695880 2548 policy_none.go:49] "None policy: Start" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.695895 2548 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.695905 2548 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.696830 2548 policy_none.go:47] "Start" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.734253 2548 manager.go:337] "Starting Device Plugin manager" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:58.734298 2548 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.734308 2548 server.go:87] "Starting device plugin registration server" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.734546 2548 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.734558 2548 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 23:02:59.417406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.734743 2548 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.734821 2548 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.734834 2548 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:58.735321 2548 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:58.735377 2548 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-106.ec2.internal\" not found" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.821225 2548 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.822528 2548 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.822562 2548 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.822588 2548 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:58.822627 2548 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.824979 2548 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.835257 2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.836476 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasSufficientMemory" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.836516 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.836532 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasSufficientPID" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.836557 2548 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.846100 2548 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:58.846121 2548 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-106.ec2.internal\": node \"ip-10-0-135-106.ec2.internal\" not found" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:58.863678 2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-106.ec2.internal\" not found" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.926662 2548 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-106.ec2.internal"] Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.926770 2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.928652 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasSufficientMemory" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.928677 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.928687 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasSufficientPID" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.930051 2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.930171 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.930198 2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.930882 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasSufficientMemory" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.930903 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.930913 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasSufficientPID" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.930924 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasSufficientMemory" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.930942 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.930953 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasSufficientPID" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.932220 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.932247 2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 23:02:59.419529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.934018 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasSufficientMemory" Jul 08 23:02:59.421401 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.934044 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 23:02:59.421401 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.934061 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeHasSufficientPID" Jul 08 23:02:59.421401 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.958064 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/641c0d2e09eac1bc2f01a3f9dd96a166-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal\" (UID: \"641c0d2e09eac1bc2f01a3f9dd96a166\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.421401 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.958101 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/641c0d2e09eac1bc2f01a3f9dd96a166-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal\" (UID: \"641c0d2e09eac1bc2f01a3f9dd96a166\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.421401 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:58.958133 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c95e074b8f38074e93586ba30700e2a9-config\") pod \"kube-apiserver-proxy-ip-10-0-135-106.ec2.internal\" (UID: \"c95e074b8f38074e93586ba30700e2a9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.421401 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:58.959506 2548 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-106.ec2.internal\" not found" node="ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.421401 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:58.964025 2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-106.ec2.internal\" not found" Jul 08 23:02:59.421401 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:58.964112 2548 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-106.ec2.internal\" not found" node="ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.421401 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.058662 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/641c0d2e09eac1bc2f01a3f9dd96a166-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal\" (UID: \"641c0d2e09eac1bc2f01a3f9dd96a166\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.421401 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.058694 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/641c0d2e09eac1bc2f01a3f9dd96a166-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal\" (UID: \"641c0d2e09eac1bc2f01a3f9dd96a166\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.421401 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.058713 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c95e074b8f38074e93586ba30700e2a9-config\") pod \"kube-apiserver-proxy-ip-10-0-135-106.ec2.internal\" (UID: \"c95e074b8f38074e93586ba30700e2a9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.421401 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.058743 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c95e074b8f38074e93586ba30700e2a9-config\") pod \"kube-apiserver-proxy-ip-10-0-135-106.ec2.internal\" (UID: \"c95e074b8f38074e93586ba30700e2a9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.421401 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.058755 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/641c0d2e09eac1bc2f01a3f9dd96a166-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal\" (UID: \"641c0d2e09eac1bc2f01a3f9dd96a166\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.421401 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.058766 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/641c0d2e09eac1bc2f01a3f9dd96a166-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal\" (UID: \"641c0d2e09eac1bc2f01a3f9dd96a166\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.421401 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:59.064116 2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-106.ec2.internal\" not found" Jul 08 23:02:59.421401 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:59.164210 2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-106.ec2.internal\" not found" Jul 08 23:02:59.421401 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.262965 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.421401 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:59.265315 2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-106.ec2.internal\" not found" Jul 08 23:02:59.423232 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.268091 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.423232 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:59.366073 2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-106.ec2.internal\" not found" Jul 08 23:02:59.466507 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:59.466476 2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-106.ec2.internal\" not found" Jul 08 23:02:59.536819 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.536787 2548 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 23:02:59.557146 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.557118 2548 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.567847 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.567826 2548 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 23:02:59.570032 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.570017 2548 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-106.ec2.internal" Jul 08 23:02:59.576039 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.576025 2548 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 23:02:59.586972 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.586947 2548 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 23:02:59.587113 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.587094 2548 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 23:02:59.587170 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.587132 2548 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 23:02:59.587170 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.587132 2548 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 23:02:59.629753 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.629706 2548 apiserver.go:52] "Watching apiserver" Jul 08 23:02:59.651234 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.651170 2548 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 22:57:59 +0000 UTC" deadline="2027-12-21 00:07:40.230520026 +0000 UTC" Jul 08 23:02:59.651234 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.651230 2548 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12721h4m40.57929528s" Jul 08 23:02:59.652481 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.652458 2548 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 23:02:59.653591 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.653568 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-mdhnl","openshift-network-diagnostics/network-check-target-chcsc","openshift-ovn-kubernetes/ovnkube-node-fv2dh","kube-system/global-pull-secret-syncer-dwldq","kube-system/konnectivity-agent-x6gwk","kube-system/kube-apiserver-proxy-ip-10-0-135-106.ec2.internal","openshift-dns/node-resolver-wnnrr","openshift-multus/multus-additional-cni-plugins-nwsqm","openshift-multus/network-metrics-daemon-4qwwh","openshift-network-operator/iptables-alerter-jv2x8","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt","openshift-cluster-node-tuning-operator/tuned-blm7c","openshift-image-registry/node-ca-mt56m","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal"] Jul 08 23:02:59.655320 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.655298 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wnnrr" Jul 08 23:02:59.656156 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.656137 2548 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 23:02:59.656466 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.656444 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mt56m" Jul 08 23:02:59.657722 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.657702 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.658068 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.658048 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 23:02:59.658158 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.658095 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ds8f2\"" Jul 08 23:02:59.658158 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.658137 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 23:02:59.658785 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.658765 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:02:59.658870 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:59.658834 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:02:59.658932 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.658872 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 23:02:59.659117 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.659103 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bf7kl\"" Jul 08 23:02:59.659352 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.659334 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 23:02:59.659432 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.659361 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 23:02:59.659822 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.659807 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x6gwk" Jul 08 23:02:59.660779 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.660761 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 23:02:59.660968 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.660953 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.661752 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.661731 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 23:02:59.661851 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.661735 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-89jz2\"" Jul 08 23:02:59.661983 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.661968 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 23:02:59.662040 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.661979 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 23:02:59.662083 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.661988 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 23:02:59.662083 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.662014 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 23:02:59.662360 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.662319 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 23:02:59.662360 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.662348 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 23:02:59.662485 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.662436 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:02:59.662541 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:59.662507 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:02:59.663109 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.663077 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-27vjp\"" Jul 08 23:02:59.663438 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.663417 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 23:02:59.663539 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.663471 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 23:02:59.663731 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.663714 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.663870 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.663802 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lblxr\"" Jul 08 23:02:59.663971 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.663934 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 23:02:59.664249 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.664227 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 23:02:59.664330 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.664293 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 23:02:59.664886 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.664865 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:02:59.665003 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.664914 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdg62\" (UniqueName: \"kubernetes.io/projected/c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28-kube-api-access-fdg62\") pod \"node-resolver-wnnrr\" (UID: \"c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28\") " pod="openshift-dns/node-resolver-wnnrr" Jul 08 23:02:59.665003 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.664963 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tvlz\" (UniqueName: \"kubernetes.io/projected/44817e25-f0f3-44fc-ab14-f1cc0cddd89e-kube-api-access-6tvlz\") pod \"node-ca-mt56m\" (UID: \"44817e25-f0f3-44fc-ab14-f1cc0cddd89e\") " pod="openshift-image-registry/node-ca-mt56m" Jul 08 23:02:59.665003 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:59.664935 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:02:59.665003 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.664991 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-run-systemd\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.665211 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665019 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-run-ovn-kubernetes\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.665211 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665042 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-cni-bin\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.665211 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665067 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28-tmp-dir\") pod \"node-resolver-wnnrr\" (UID: \"c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28\") " pod="openshift-dns/node-resolver-wnnrr" Jul 08 23:02:59.665211 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665090 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-run-ovn\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.665211 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665115 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-env-overrides\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.665211 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665139 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-ovnkube-script-lib\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.665211 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665164 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-etc-openvswitch\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.665211 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665180 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-run-openvswitch\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.665211 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665201 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-ovn-node-metrics-cert\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.665577 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665239 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs6cq\" (UniqueName: \"kubernetes.io/projected/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-kube-api-access-qs6cq\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.665577 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665280 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28-hosts-file\") pod \"node-resolver-wnnrr\" (UID: \"c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28\") " pod="openshift-dns/node-resolver-wnnrr" Jul 08 23:02:59.665577 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665316 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/44817e25-f0f3-44fc-ab14-f1cc0cddd89e-host\") pod \"node-ca-mt56m\" (UID: \"44817e25-f0f3-44fc-ab14-f1cc0cddd89e\") " pod="openshift-image-registry/node-ca-mt56m" Jul 08 23:02:59.665577 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665341 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-cni-netd\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.665577 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665382 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.665577 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665430 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/44817e25-f0f3-44fc-ab14-f1cc0cddd89e-serviceca\") pod \"node-ca-mt56m\" (UID: \"44817e25-f0f3-44fc-ab14-f1cc0cddd89e\") " pod="openshift-image-registry/node-ca-mt56m" Jul 08 23:02:59.665577 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665458 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-kubelet\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.665577 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665490 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-systemd-units\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.665577 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665515 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-run-netns\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.665577 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665532 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-ovnkube-config\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.665577 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665547 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-var-lib-openvswitch\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.665577 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665562 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-node-log\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.666137 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665595 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-log-socket\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.666137 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.665655 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-slash\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.666137 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.666076 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 23:02:59.666276 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.666194 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jv2x8" Jul 08 23:02:59.666380 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.666361 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-v4j9b\"" Jul 08 23:02:59.667874 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.667568 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.669230 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.668967 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5d62l\"" Jul 08 23:02:59.669230 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.669003 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.669230 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.669036 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 23:02:59.669230 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.669064 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 23:02:59.669522 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.669244 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 23:02:59.669649 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.669625 2548 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 23:02:59.670073 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.670032 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 23:02:59.671039 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.670506 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 23:02:59.671039 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.670787 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 23:02:59.671039 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.670857 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vncq8\"" Jul 08 23:02:59.672223 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.671954 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 23:02:59.672321 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.672254 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 23:02:59.672851 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.672556 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9cfj5\"" Jul 08 23:02:59.691366 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.691339 2548 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hzqnp" Jul 08 23:02:59.699409 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.699391 2548 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hzqnp" Jul 08 23:02:59.758262 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.758235 2548 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 23:02:59.765985 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.765961 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-ovnkube-script-lib\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.766109 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766003 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40709248-e1df-4adb-ac19-9be50f5cf61b-tmp\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.766109 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766032 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/58690598-bf03-4609-bc53-3d4b5edca7fc-system-cni-dir\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.766109 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766058 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/58690598-bf03-4609-bc53-3d4b5edca7fc-os-release\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.766109 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766085 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-etc-openvswitch\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.766291 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766110 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-run-openvswitch\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.766291 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766137 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-ovn-node-metrics-cert\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.766291 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766162 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-etc-openvswitch\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.766291 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766165 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9984b179-b525-4ea8-b339-be8b23fc35b2-cni-binary-copy\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.766291 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766194 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-run-openvswitch\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.766291 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766214 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-sysctl-d\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.766291 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766238 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-sys\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.766291 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766256 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfwff\" (UniqueName: \"kubernetes.io/projected/40709248-e1df-4adb-ac19-9be50f5cf61b-kube-api-access-bfwff\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.766291 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766273 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs\") pod \"network-metrics-daemon-4qwwh\" (UID: \"48abd27d-5a99-4c70-937b-b9d06f033843\") " pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:02:59.767496 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766297 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/44817e25-f0f3-44fc-ab14-f1cc0cddd89e-host\") pod \"node-ca-mt56m\" (UID: \"44817e25-f0f3-44fc-ab14-f1cc0cddd89e\") " pod="openshift-image-registry/node-ca-mt56m" Jul 08 23:02:59.767496 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766313 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-etc-kubernetes\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.767496 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766329 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-sysconfig\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.767496 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766349 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-lib-modules\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.767496 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766374 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3345846f-189a-4f26-bd90-737c47f0f938-iptables-alerter-script\") pod \"iptables-alerter-jv2x8\" (UID: \"3345846f-189a-4f26-bd90-737c47f0f938\") " pod="openshift-network-operator/iptables-alerter-jv2x8" Jul 08 23:02:59.767496 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766380 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/44817e25-f0f3-44fc-ab14-f1cc0cddd89e-host\") pod \"node-ca-mt56m\" (UID: \"44817e25-f0f3-44fc-ab14-f1cc0cddd89e\") " pod="openshift-image-registry/node-ca-mt56m" Jul 08 23:02:59.767496 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766414 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/608dffea-c3de-4379-9289-433428dc33f0-sys-fs\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.767496 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766451 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/213525b4-d967-437a-9c90-d40f2190a5a5-kubelet-config\") pod \"global-pull-secret-syncer-dwldq\" (UID: \"213525b4-d967-437a-9c90-d40f2190a5a5\") " pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:02:59.767496 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766482 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret\") pod \"global-pull-secret-syncer-dwldq\" (UID: \"213525b4-d967-437a-9c90-d40f2190a5a5\") " pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:02:59.767496 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766506 2548 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 23:02:59.767496 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766515 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-kubelet\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.767496 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766562 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-host-run-netns\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.767496 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766584 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-kubelet\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.767496 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766591 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/608dffea-c3de-4379-9289-433428dc33f0-registration-dir\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.768238 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766616 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/58690598-bf03-4609-bc53-3d4b5edca7fc-cni-binary-copy\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.768238 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766629 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-ovnkube-script-lib\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.768238 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766642 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/58690598-bf03-4609-bc53-3d4b5edca7fc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.768238 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766708 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-run-netns\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.768238 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766732 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-host-var-lib-kubelet\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.768238 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766760 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t67kc\" (UniqueName: \"kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc\") pod \"network-check-target-chcsc\" (UID: \"825c51de-1ee6-48cd-8e9c-66811e605fbf\") " pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:02:59.768238 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766776 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/608dffea-c3de-4379-9289-433428dc33f0-etc-selinux\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.768238 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766786 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-run-netns\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.768238 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766801 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/58690598-bf03-4609-bc53-3d4b5edca7fc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.768238 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766825 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/58690598-bf03-4609-bc53-3d4b5edca7fc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.768238 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766848 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-var-lib-openvswitch\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.768238 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766863 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-host-var-lib-cni-bin\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.768238 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766878 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-slash\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.768860 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766914 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-system-cni-dir\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.768860 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766925 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-slash\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.768860 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766925 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-var-lib-openvswitch\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.768860 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766960 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-multus-socket-dir-parent\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.768860 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.766978 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-sysctl-conf\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.768860 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767011 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-host\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.768860 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767045 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3345846f-189a-4f26-bd90-737c47f0f938-host-slash\") pod \"iptables-alerter-jv2x8\" (UID: \"3345846f-189a-4f26-bd90-737c47f0f938\") " pod="openshift-network-operator/iptables-alerter-jv2x8" Jul 08 23:02:59.768860 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767070 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/608dffea-c3de-4379-9289-433428dc33f0-socket-dir\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.768860 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767086 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/608dffea-c3de-4379-9289-433428dc33f0-device-dir\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.768860 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767104 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-run-systemd\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.768860 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767124 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-run-ovn-kubernetes\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.768860 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767148 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-host-var-lib-cni-multus\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.768860 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767159 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-run-systemd\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.768860 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767174 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-hostroot\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.769574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767190 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-run-ovn-kubernetes\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.769574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767197 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-run-ovn\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.769574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767213 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-env-overrides\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.769574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767228 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzrmm\" (UniqueName: \"kubernetes.io/projected/9984b179-b525-4ea8-b339-be8b23fc35b2-kube-api-access-nzrmm\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.769574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767244 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-kubernetes\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.769574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767267 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-tuned\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.769574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767286 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-run-ovn\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.769574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767316 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs6cq\" (UniqueName: \"kubernetes.io/projected/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-kube-api-access-qs6cq\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.769574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767347 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-host-run-k8s-cni-cncf-io\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.769574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767376 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-host-run-multus-certs\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.769574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767475 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28-hosts-file\") pod \"node-resolver-wnnrr\" (UID: \"c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28\") " pod="openshift-dns/node-resolver-wnnrr" Jul 08 23:02:59.769574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767494 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-cni-netd\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.769574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767549 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.769574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767565 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-env-overrides\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.770191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767549 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28-hosts-file\") pod \"node-resolver-wnnrr\" (UID: \"c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28\") " pod="openshift-dns/node-resolver-wnnrr" Jul 08 23:02:59.770191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767593 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-cni-netd\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.770191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767584 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9984b179-b525-4ea8-b339-be8b23fc35b2-multus-daemon-config\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.770191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767636 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.770191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767644 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/44817e25-f0f3-44fc-ab14-f1cc0cddd89e-serviceca\") pod \"node-ca-mt56m\" (UID: \"44817e25-f0f3-44fc-ab14-f1cc0cddd89e\") " pod="openshift-image-registry/node-ca-mt56m" Jul 08 23:02:59.770191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767678 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-modprobe-d\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.770191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767699 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-systemd-units\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.770191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767717 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-ovnkube-config\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.770191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767741 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-cnibin\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.770191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767745 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-systemd-units\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.770191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767764 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-multus-conf-dir\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.770191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767801 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-systemd\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.770191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767834 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/608dffea-c3de-4379-9289-433428dc33f0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.770191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767862 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aa3acccd-8292-46b0-b34b-d6ecba8fc595-agent-certs\") pod \"konnectivity-agent-x6gwk\" (UID: \"aa3acccd-8292-46b0-b34b-d6ecba8fc595\") " pod="kube-system/konnectivity-agent-x6gwk" Jul 08 23:02:59.770596 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767888 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-node-log\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.770596 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767914 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-log-socket\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.770596 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767936 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-multus-cni-dir\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.770596 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767962 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-run\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.770596 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767965 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-node-log\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.770596 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767987 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-var-lib-kubelet\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.770596 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.767997 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-log-socket\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.770596 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.768012 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgx84\" (UniqueName: \"kubernetes.io/projected/608dffea-c3de-4379-9289-433428dc33f0-kube-api-access-tgx84\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.770596 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.768038 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-os-release\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.770596 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.768056 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/44817e25-f0f3-44fc-ab14-f1cc0cddd89e-serviceca\") pod \"node-ca-mt56m\" (UID: \"44817e25-f0f3-44fc-ab14-f1cc0cddd89e\") " pod="openshift-image-registry/node-ca-mt56m" Jul 08 23:02:59.770596 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.768063 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4vzc\" (UniqueName: \"kubernetes.io/projected/3345846f-189a-4f26-bd90-737c47f0f938-kube-api-access-t4vzc\") pod \"iptables-alerter-jv2x8\" (UID: \"3345846f-189a-4f26-bd90-737c47f0f938\") " pod="openshift-network-operator/iptables-alerter-jv2x8" Jul 08 23:02:59.770596 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.768087 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aa3acccd-8292-46b0-b34b-d6ecba8fc595-konnectivity-ca\") pod \"konnectivity-agent-x6gwk\" (UID: \"aa3acccd-8292-46b0-b34b-d6ecba8fc595\") " pod="kube-system/konnectivity-agent-x6gwk" Jul 08 23:02:59.770596 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.768124 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdg62\" (UniqueName: \"kubernetes.io/projected/c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28-kube-api-access-fdg62\") pod \"node-resolver-wnnrr\" (UID: \"c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28\") " pod="openshift-dns/node-resolver-wnnrr" Jul 08 23:02:59.770596 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.768153 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tvlz\" (UniqueName: \"kubernetes.io/projected/44817e25-f0f3-44fc-ab14-f1cc0cddd89e-kube-api-access-6tvlz\") pod \"node-ca-mt56m\" (UID: \"44817e25-f0f3-44fc-ab14-f1cc0cddd89e\") " pod="openshift-image-registry/node-ca-mt56m" Jul 08 23:02:59.771028 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.768178 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-cni-bin\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.771028 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.768204 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ttmv\" (UniqueName: \"kubernetes.io/projected/48abd27d-5a99-4c70-937b-b9d06f033843-kube-api-access-7ttmv\") pod \"network-metrics-daemon-4qwwh\" (UID: \"48abd27d-5a99-4c70-937b-b9d06f033843\") " pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:02:59.771028 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.768219 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-ovnkube-config\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.771028 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.768231 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/213525b4-d967-437a-9c90-d40f2190a5a5-dbus\") pod \"global-pull-secret-syncer-dwldq\" (UID: \"213525b4-d967-437a-9c90-d40f2190a5a5\") " pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:02:59.771028 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.768238 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-host-cni-bin\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.771028 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.768272 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/58690598-bf03-4609-bc53-3d4b5edca7fc-cnibin\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.771028 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.768321 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q4w5\" (UniqueName: \"kubernetes.io/projected/58690598-bf03-4609-bc53-3d4b5edca7fc-kube-api-access-2q4w5\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.771028 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.768353 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28-tmp-dir\") pod \"node-resolver-wnnrr\" (UID: \"c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28\") " pod="openshift-dns/node-resolver-wnnrr" Jul 08 23:02:59.771028 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.768589 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28-tmp-dir\") pod \"node-resolver-wnnrr\" (UID: \"c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28\") " pod="openshift-dns/node-resolver-wnnrr" Jul 08 23:02:59.771028 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.770216 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-ovn-node-metrics-cert\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.777402 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.777343 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs6cq\" (UniqueName: \"kubernetes.io/projected/b5236fd2-5d92-4b80-afe4-68b9bbb567b8-kube-api-access-qs6cq\") pod \"ovnkube-node-fv2dh\" (UID: \"b5236fd2-5d92-4b80-afe4-68b9bbb567b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:02:59.777509 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.777453 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tvlz\" (UniqueName: \"kubernetes.io/projected/44817e25-f0f3-44fc-ab14-f1cc0cddd89e-kube-api-access-6tvlz\") pod \"node-ca-mt56m\" (UID: \"44817e25-f0f3-44fc-ab14-f1cc0cddd89e\") " pod="openshift-image-registry/node-ca-mt56m" Jul 08 23:02:59.777903 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.777882 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdg62\" (UniqueName: \"kubernetes.io/projected/c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28-kube-api-access-fdg62\") pod \"node-resolver-wnnrr\" (UID: \"c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28\") " pod="openshift-dns/node-resolver-wnnrr" Jul 08 23:02:59.802421 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:59.802384 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc95e074b8f38074e93586ba30700e2a9.slice/crio-fd09696dd558300bd1e432c3f324a1769203468fb08bdd58010f32a6481649b8 WatchSource:0}: Error finding container fd09696dd558300bd1e432c3f324a1769203468fb08bdd58010f32a6481649b8: Status 404 returned error can't find the container with id fd09696dd558300bd1e432c3f324a1769203468fb08bdd58010f32a6481649b8 Jul 08 23:02:59.802908 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:59.802887 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod641c0d2e09eac1bc2f01a3f9dd96a166.slice/crio-e54ba35d3b773e3c1c92fdf92786a95e9a3d87eb1248e6f280f417e28c2ce50f WatchSource:0}: Error finding container e54ba35d3b773e3c1c92fdf92786a95e9a3d87eb1248e6f280f417e28c2ce50f: Status 404 returned error can't find the container with id e54ba35d3b773e3c1c92fdf92786a95e9a3d87eb1248e6f280f417e28c2ce50f Jul 08 23:02:59.808943 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.808870 2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 23:02:59.826116 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.826049 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal" event={"ID":"641c0d2e09eac1bc2f01a3f9dd96a166","Type":"ContainerStarted","Data":"e54ba35d3b773e3c1c92fdf92786a95e9a3d87eb1248e6f280f417e28c2ce50f"} Jul 08 23:02:59.827177 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.827154 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-106.ec2.internal" event={"ID":"c95e074b8f38074e93586ba30700e2a9","Type":"ContainerStarted","Data":"fd09696dd558300bd1e432c3f324a1769203468fb08bdd58010f32a6481649b8"} Jul 08 23:02:59.868737 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.868700 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-host-var-lib-kubelet\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.868893 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.868759 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t67kc\" (UniqueName: \"kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc\") pod \"network-check-target-chcsc\" (UID: \"825c51de-1ee6-48cd-8e9c-66811e605fbf\") " pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:02:59.868893 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.868788 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/608dffea-c3de-4379-9289-433428dc33f0-etc-selinux\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.868893 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.868806 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/58690598-bf03-4609-bc53-3d4b5edca7fc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.868893 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.868811 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-host-var-lib-kubelet\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.868893 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.868833 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/58690598-bf03-4609-bc53-3d4b5edca7fc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.868893 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.868852 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-host-var-lib-cni-bin\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.868893 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.868869 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-system-cni-dir\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.868893 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.868886 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-multus-socket-dir-parent\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.869356 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.868904 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-sysctl-conf\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.869356 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.868926 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-host\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.869356 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.868949 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3345846f-189a-4f26-bd90-737c47f0f938-host-slash\") pod \"iptables-alerter-jv2x8\" (UID: \"3345846f-189a-4f26-bd90-737c47f0f938\") " pod="openshift-network-operator/iptables-alerter-jv2x8" Jul 08 23:02:59.869356 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.868973 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/608dffea-c3de-4379-9289-433428dc33f0-socket-dir\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.869356 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.868953 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-host-var-lib-cni-bin\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.869356 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.868996 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/608dffea-c3de-4379-9289-433428dc33f0-device-dir\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.869356 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869048 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/608dffea-c3de-4379-9289-433428dc33f0-device-dir\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.869356 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869070 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-system-cni-dir\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.869356 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869111 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-host-var-lib-cni-multus\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.869356 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869113 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-sysctl-conf\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.869356 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869070 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-host-var-lib-cni-multus\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.869356 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869163 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-host\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.869356 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869011 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-multus-socket-dir-parent\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.869356 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869186 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-hostroot\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.869356 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869214 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/608dffea-c3de-4379-9289-433428dc33f0-socket-dir\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.870105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869215 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzrmm\" (UniqueName: \"kubernetes.io/projected/9984b179-b525-4ea8-b339-be8b23fc35b2-kube-api-access-nzrmm\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.870105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869223 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3345846f-189a-4f26-bd90-737c47f0f938-host-slash\") pod \"iptables-alerter-jv2x8\" (UID: \"3345846f-189a-4f26-bd90-737c47f0f938\") " pod="openshift-network-operator/iptables-alerter-jv2x8" Jul 08 23:02:59.870105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869222 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/608dffea-c3de-4379-9289-433428dc33f0-etc-selinux\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.870105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869250 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-kubernetes\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.870105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869257 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-hostroot\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.870105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869276 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-tuned\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.870105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869314 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-host-run-k8s-cni-cncf-io\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.870105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869337 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-kubernetes\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.870105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869345 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-host-run-multus-certs\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.870105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869366 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-host-run-k8s-cni-cncf-io\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.870105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869382 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-host-run-multus-certs\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.870105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869387 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9984b179-b525-4ea8-b339-be8b23fc35b2-multus-daemon-config\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.870105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869425 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-modprobe-d\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.870105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869452 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-cnibin\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.870105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869522 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-multus-conf-dir\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.870916 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869529 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/58690598-bf03-4609-bc53-3d4b5edca7fc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.870916 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869591 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-cnibin\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.870916 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869555 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/58690598-bf03-4609-bc53-3d4b5edca7fc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.870916 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869588 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-modprobe-d\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.870916 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869628 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-multus-conf-dir\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.870916 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869549 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-systemd\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.870916 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869671 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/608dffea-c3de-4379-9289-433428dc33f0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.870916 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869687 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-systemd\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.870916 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869709 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aa3acccd-8292-46b0-b34b-d6ecba8fc595-agent-certs\") pod \"konnectivity-agent-x6gwk\" (UID: \"aa3acccd-8292-46b0-b34b-d6ecba8fc595\") " pod="kube-system/konnectivity-agent-x6gwk" Jul 08 23:02:59.870916 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869727 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-multus-cni-dir\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.870916 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869745 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-run\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.870916 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869746 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/608dffea-c3de-4379-9289-433428dc33f0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.870916 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869793 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-var-lib-kubelet\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.870916 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869800 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-run\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.870916 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869807 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-multus-cni-dir\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.871704 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869825 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgx84\" (UniqueName: \"kubernetes.io/projected/608dffea-c3de-4379-9289-433428dc33f0-kube-api-access-tgx84\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.871704 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869861 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-var-lib-kubelet\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.871704 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.869884 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-os-release\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.871704 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870002 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4vzc\" (UniqueName: \"kubernetes.io/projected/3345846f-189a-4f26-bd90-737c47f0f938-kube-api-access-t4vzc\") pod \"iptables-alerter-jv2x8\" (UID: \"3345846f-189a-4f26-bd90-737c47f0f938\") " pod="openshift-network-operator/iptables-alerter-jv2x8" Jul 08 23:02:59.871704 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870035 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aa3acccd-8292-46b0-b34b-d6ecba8fc595-konnectivity-ca\") pod \"konnectivity-agent-x6gwk\" (UID: \"aa3acccd-8292-46b0-b34b-d6ecba8fc595\") " pod="kube-system/konnectivity-agent-x6gwk" Jul 08 23:02:59.871704 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870046 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-os-release\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.871704 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870062 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ttmv\" (UniqueName: \"kubernetes.io/projected/48abd27d-5a99-4c70-937b-b9d06f033843-kube-api-access-7ttmv\") pod \"network-metrics-daemon-4qwwh\" (UID: \"48abd27d-5a99-4c70-937b-b9d06f033843\") " pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:02:59.871704 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870080 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/213525b4-d967-437a-9c90-d40f2190a5a5-dbus\") pod \"global-pull-secret-syncer-dwldq\" (UID: \"213525b4-d967-437a-9c90-d40f2190a5a5\") " pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:02:59.871704 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870103 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/58690598-bf03-4609-bc53-3d4b5edca7fc-cnibin\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.871704 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870127 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q4w5\" (UniqueName: \"kubernetes.io/projected/58690598-bf03-4609-bc53-3d4b5edca7fc-kube-api-access-2q4w5\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.871704 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870138 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9984b179-b525-4ea8-b339-be8b23fc35b2-multus-daemon-config\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.871704 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870164 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40709248-e1df-4adb-ac19-9be50f5cf61b-tmp\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.871704 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870189 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/58690598-bf03-4609-bc53-3d4b5edca7fc-system-cni-dir\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.871704 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870200 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/58690598-bf03-4609-bc53-3d4b5edca7fc-cnibin\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.872374 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870213 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/58690598-bf03-4609-bc53-3d4b5edca7fc-os-release\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.872374 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870242 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9984b179-b525-4ea8-b339-be8b23fc35b2-cni-binary-copy\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.872374 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870245 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/58690598-bf03-4609-bc53-3d4b5edca7fc-system-cni-dir\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.872374 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870269 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-sysctl-d\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.872374 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870293 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-sys\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.872374 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870317 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfwff\" (UniqueName: \"kubernetes.io/projected/40709248-e1df-4adb-ac19-9be50f5cf61b-kube-api-access-bfwff\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.872374 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870344 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs\") pod \"network-metrics-daemon-4qwwh\" (UID: \"48abd27d-5a99-4c70-937b-b9d06f033843\") " pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:02:59.872374 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870377 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-etc-kubernetes\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.872374 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870402 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-sysconfig\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.872374 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870430 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-lib-modules\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.872374 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870456 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3345846f-189a-4f26-bd90-737c47f0f938-iptables-alerter-script\") pod \"iptables-alerter-jv2x8\" (UID: \"3345846f-189a-4f26-bd90-737c47f0f938\") " pod="openshift-network-operator/iptables-alerter-jv2x8" Jul 08 23:02:59.872374 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870481 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/608dffea-c3de-4379-9289-433428dc33f0-sys-fs\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.872374 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870506 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/213525b4-d967-437a-9c90-d40f2190a5a5-kubelet-config\") pod \"global-pull-secret-syncer-dwldq\" (UID: \"213525b4-d967-437a-9c90-d40f2190a5a5\") " pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:02:59.872374 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870534 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret\") pod \"global-pull-secret-syncer-dwldq\" (UID: \"213525b4-d967-437a-9c90-d40f2190a5a5\") " pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:02:59.872958 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870563 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-host-run-netns\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.872958 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870590 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/608dffea-c3de-4379-9289-433428dc33f0-registration-dir\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.872958 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870636 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/58690598-bf03-4609-bc53-3d4b5edca7fc-cni-binary-copy\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.872958 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870685 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/58690598-bf03-4609-bc53-3d4b5edca7fc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.872958 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870702 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aa3acccd-8292-46b0-b34b-d6ecba8fc595-konnectivity-ca\") pod \"konnectivity-agent-x6gwk\" (UID: \"aa3acccd-8292-46b0-b34b-d6ecba8fc595\") " pod="kube-system/konnectivity-agent-x6gwk" Jul 08 23:02:59.872958 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870869 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/58690598-bf03-4609-bc53-3d4b5edca7fc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.872958 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870876 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/58690598-bf03-4609-bc53-3d4b5edca7fc-os-release\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.872958 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.870992 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-sysctl-d\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.872958 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.871042 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-sys\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.872958 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.871103 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9984b179-b525-4ea8-b339-be8b23fc35b2-cni-binary-copy\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.872958 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.871196 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-host-run-netns\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.872958 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.871200 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/608dffea-c3de-4379-9289-433428dc33f0-sys-fs\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.872958 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.871256 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/213525b4-d967-437a-9c90-d40f2190a5a5-kubelet-config\") pod \"global-pull-secret-syncer-dwldq\" (UID: \"213525b4-d967-437a-9c90-d40f2190a5a5\") " pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:02:59.872958 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.871269 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/608dffea-c3de-4379-9289-433428dc33f0-registration-dir\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.872958 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:59.871305 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 23:02:59.873514 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:59.871366 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 23:02:59.873514 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:59.871372 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs podName:48abd27d-5a99-4c70-937b-b9d06f033843 nodeName:}" failed. No retries permitted until 2026-07-08 23:03:00.371352106 +0000 UTC m=+2.131847162 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs") pod "network-metrics-daemon-4qwwh" (UID: "48abd27d-5a99-4c70-937b-b9d06f033843") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 23:02:59.873514 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:59.871440 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret podName:213525b4-d967-437a-9c90-d40f2190a5a5 nodeName:}" failed. No retries permitted until 2026-07-08 23:03:00.371422372 +0000 UTC m=+2.131917422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret") pod "global-pull-secret-syncer-dwldq" (UID: "213525b4-d967-437a-9c90-d40f2190a5a5") : object "kube-system"/"original-pull-secret" not registered Jul 08 23:02:59.873514 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.871551 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-lib-modules\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.873514 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.871613 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9984b179-b525-4ea8-b339-be8b23fc35b2-etc-kubernetes\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.873514 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.871656 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-sysconfig\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.873514 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.871777 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/58690598-bf03-4609-bc53-3d4b5edca7fc-cni-binary-copy\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.873514 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.871849 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/213525b4-d967-437a-9c90-d40f2190a5a5-dbus\") pod \"global-pull-secret-syncer-dwldq\" (UID: \"213525b4-d967-437a-9c90-d40f2190a5a5\") " pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:02:59.873514 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.872022 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3345846f-189a-4f26-bd90-737c47f0f938-iptables-alerter-script\") pod \"iptables-alerter-jv2x8\" (UID: \"3345846f-189a-4f26-bd90-737c47f0f938\") " pod="openshift-network-operator/iptables-alerter-jv2x8" Jul 08 23:02:59.873514 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.872159 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/40709248-e1df-4adb-ac19-9be50f5cf61b-etc-tuned\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.873514 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.872434 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aa3acccd-8292-46b0-b34b-d6ecba8fc595-agent-certs\") pod \"konnectivity-agent-x6gwk\" (UID: \"aa3acccd-8292-46b0-b34b-d6ecba8fc595\") " pod="kube-system/konnectivity-agent-x6gwk" Jul 08 23:02:59.873514 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.872574 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40709248-e1df-4adb-ac19-9be50f5cf61b-tmp\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.875272 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:59.875255 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 23:02:59.875346 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:59.875276 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 23:02:59.875346 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:59.875286 2548 projected.go:196] Error preparing data for projected volume kube-api-access-t67kc for pod openshift-network-diagnostics/network-check-target-chcsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 23:02:59.875425 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:02:59.875400 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc podName:825c51de-1ee6-48cd-8e9c-66811e605fbf nodeName:}" failed. No retries permitted until 2026-07-08 23:03:00.375388567 +0000 UTC m=+2.135883606 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-t67kc" (UniqueName: "kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc") pod "network-check-target-chcsc" (UID: "825c51de-1ee6-48cd-8e9c-66811e605fbf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 23:02:59.877132 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.877112 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzrmm\" (UniqueName: \"kubernetes.io/projected/9984b179-b525-4ea8-b339-be8b23fc35b2-kube-api-access-nzrmm\") pod \"multus-mdhnl\" (UID: \"9984b179-b525-4ea8-b339-be8b23fc35b2\") " pod="openshift-multus/multus-mdhnl" Jul 08 23:02:59.877418 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.877399 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgx84\" (UniqueName: \"kubernetes.io/projected/608dffea-c3de-4379-9289-433428dc33f0-kube-api-access-tgx84\") pod \"aws-ebs-csi-driver-node-r76jt\" (UID: \"608dffea-c3de-4379-9289-433428dc33f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:02:59.879052 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.879021 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q4w5\" (UniqueName: \"kubernetes.io/projected/58690598-bf03-4609-bc53-3d4b5edca7fc-kube-api-access-2q4w5\") pod \"multus-additional-cni-plugins-nwsqm\" (UID: \"58690598-bf03-4609-bc53-3d4b5edca7fc\") " pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:02:59.879899 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.879869 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfwff\" (UniqueName: \"kubernetes.io/projected/40709248-e1df-4adb-ac19-9be50f5cf61b-kube-api-access-bfwff\") pod \"tuned-blm7c\" (UID: \"40709248-e1df-4adb-ac19-9be50f5cf61b\") " pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:02:59.880987 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.880965 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4vzc\" (UniqueName: \"kubernetes.io/projected/3345846f-189a-4f26-bd90-737c47f0f938-kube-api-access-t4vzc\") pod \"iptables-alerter-jv2x8\" (UID: \"3345846f-189a-4f26-bd90-737c47f0f938\") " pod="openshift-network-operator/iptables-alerter-jv2x8" Jul 08 23:02:59.881054 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.881032 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ttmv\" (UniqueName: \"kubernetes.io/projected/48abd27d-5a99-4c70-937b-b9d06f033843-kube-api-access-7ttmv\") pod \"network-metrics-daemon-4qwwh\" (UID: \"48abd27d-5a99-4c70-937b-b9d06f033843\") " pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:02:59.990361 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:02:59.990278 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wnnrr" Jul 08 23:02:59.998696 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:02:59.998652 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8cf1ec4_b117_47ba_ad3e_0c0e147a2a28.slice/crio-dea5ef98d2a49b3aaaa6335425258bce8aa2c112eb2ecb377baf276d9222273a WatchSource:0}: Error finding container dea5ef98d2a49b3aaaa6335425258bce8aa2c112eb2ecb377baf276d9222273a: Status 404 returned error can't find the container with id dea5ef98d2a49b3aaaa6335425258bce8aa2c112eb2ecb377baf276d9222273a Jul 08 23:03:00.009147 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.009126 2548 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 23:03:00.013755 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.013734 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mt56m" Jul 08 23:03:00.020736 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:03:00.020706 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44817e25_f0f3_44fc_ab14_f1cc0cddd89e.slice/crio-43bd494ff052c6295506142a59f4cf92040e9bf3bd0d41f3e48ed3fed840bd11 WatchSource:0}: Error finding container 43bd494ff052c6295506142a59f4cf92040e9bf3bd0d41f3e48ed3fed840bd11: Status 404 returned error can't find the container with id 43bd494ff052c6295506142a59f4cf92040e9bf3bd0d41f3e48ed3fed840bd11 Jul 08 23:03:00.024896 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.024876 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:03:00.029693 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.029671 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x6gwk" Jul 08 23:03:00.032459 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:03:00.032437 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5236fd2_5d92_4b80_afe4_68b9bbb567b8.slice/crio-78749f2f5916d2b4a008545c5ec42611c7ed3343c155be7920881aee3d04673e WatchSource:0}: Error finding container 78749f2f5916d2b4a008545c5ec42611c7ed3343c155be7920881aee3d04673e: Status 404 returned error can't find the container with id 78749f2f5916d2b4a008545c5ec42611c7ed3343c155be7920881aee3d04673e Jul 08 23:03:00.037247 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:03:00.037228 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa3acccd_8292_46b0_b34b_d6ecba8fc595.slice/crio-8a942218af1de610e17374d9c593f9103c8c9c1a5c9dd47411017cca5da400cf WatchSource:0}: Error finding container 8a942218af1de610e17374d9c593f9103c8c9c1a5c9dd47411017cca5da400cf: Status 404 returned error can't find the container with id 8a942218af1de610e17374d9c593f9103c8c9c1a5c9dd47411017cca5da400cf Jul 08 23:03:00.042415 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.042396 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nwsqm" Jul 08 23:03:00.049972 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:03:00.049951 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58690598_bf03_4609_bc53_3d4b5edca7fc.slice/crio-a160b80cddad9a0a9390cf01dcd2aea2f39dcf1fd78e3ffc0b50c451bf8e896f WatchSource:0}: Error finding container a160b80cddad9a0a9390cf01dcd2aea2f39dcf1fd78e3ffc0b50c451bf8e896f: Status 404 returned error can't find the container with id a160b80cddad9a0a9390cf01dcd2aea2f39dcf1fd78e3ffc0b50c451bf8e896f Jul 08 23:03:00.056403 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.056387 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mdhnl" Jul 08 23:03:00.062549 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.062532 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jv2x8" Jul 08 23:03:00.063378 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:03:00.063359 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9984b179_b525_4ea8_b339_be8b23fc35b2.slice/crio-aaab9548586d5e5493fb798f8daa358f97b593527c25cd9872e52df4d499a42e WatchSource:0}: Error finding container aaab9548586d5e5493fb798f8daa358f97b593527c25cd9872e52df4d499a42e: Status 404 returned error can't find the container with id aaab9548586d5e5493fb798f8daa358f97b593527c25cd9872e52df4d499a42e Jul 08 23:03:00.070029 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:03:00.070008 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3345846f_189a_4f26_bd90_737c47f0f938.slice/crio-94eee795992b8c9a3f22d072e06e7a9fd2e8be2d7145e8aa1d76a750de933c28 WatchSource:0}: Error finding container 94eee795992b8c9a3f22d072e06e7a9fd2e8be2d7145e8aa1d76a750de933c28: Status 404 returned error can't find the container with id 94eee795992b8c9a3f22d072e06e7a9fd2e8be2d7145e8aa1d76a750de933c28 Jul 08 23:03:00.087530 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.087509 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" Jul 08 23:03:00.091943 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.091922 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-blm7c" Jul 08 23:03:00.095853 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:03:00.095833 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod608dffea_c3de_4379_9289_433428dc33f0.slice/crio-38660bf8426c3a223a0f7958ed6db19beab55bc86bf87fd0470f6cf61d6f26aa WatchSource:0}: Error finding container 38660bf8426c3a223a0f7958ed6db19beab55bc86bf87fd0470f6cf61d6f26aa: Status 404 returned error can't find the container with id 38660bf8426c3a223a0f7958ed6db19beab55bc86bf87fd0470f6cf61d6f26aa Jul 08 23:03:00.100581 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:03:00.100558 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40709248_e1df_4adb_ac19_9be50f5cf61b.slice/crio-f538cddab3a75067d4bbfa232aaa20dc56578460ebeeb0a67afc9497cbbd66b8 WatchSource:0}: Error finding container f538cddab3a75067d4bbfa232aaa20dc56578460ebeeb0a67afc9497cbbd66b8: Status 404 returned error can't find the container with id f538cddab3a75067d4bbfa232aaa20dc56578460ebeeb0a67afc9497cbbd66b8 Jul 08 23:03:00.374412 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.374014 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs\") pod \"network-metrics-daemon-4qwwh\" (UID: \"48abd27d-5a99-4c70-937b-b9d06f033843\") " pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:00.374412 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.374073 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret\") pod \"global-pull-secret-syncer-dwldq\" (UID: \"213525b4-d967-437a-9c90-d40f2190a5a5\") " pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:00.374412 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:00.374196 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 23:03:00.374412 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:00.374197 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 23:03:00.374412 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:00.374277 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret podName:213525b4-d967-437a-9c90-d40f2190a5a5 nodeName:}" failed. No retries permitted until 2026-07-08 23:03:01.374256407 +0000 UTC m=+3.134751445 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret") pod "global-pull-secret-syncer-dwldq" (UID: "213525b4-d967-437a-9c90-d40f2190a5a5") : object "kube-system"/"original-pull-secret" not registered Jul 08 23:03:00.374412 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:00.374328 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs podName:48abd27d-5a99-4c70-937b-b9d06f033843 nodeName:}" failed. No retries permitted until 2026-07-08 23:03:01.374310699 +0000 UTC m=+3.134805746 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs") pod "network-metrics-daemon-4qwwh" (UID: "48abd27d-5a99-4c70-937b-b9d06f033843") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 23:03:00.474933 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.474895 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t67kc\" (UniqueName: \"kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc\") pod \"network-check-target-chcsc\" (UID: \"825c51de-1ee6-48cd-8e9c-66811e605fbf\") " pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:00.475109 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:00.475070 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 23:03:00.475109 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:00.475092 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 23:03:00.475109 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:00.475104 2548 projected.go:196] Error preparing data for projected volume kube-api-access-t67kc for pod openshift-network-diagnostics/network-check-target-chcsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 23:03:00.475289 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:00.475167 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc podName:825c51de-1ee6-48cd-8e9c-66811e605fbf nodeName:}" failed. No retries permitted until 2026-07-08 23:03:01.475147443 +0000 UTC m=+3.235642483 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-t67kc" (UniqueName: "kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc") pod "network-check-target-chcsc" (UID: "825c51de-1ee6-48cd-8e9c-66811e605fbf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 23:03:00.475451 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.475431 2548 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 23:03:00.582699 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.582502 2548 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 23:03:00.702535 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.702356 2548 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 22:58:00 +0000 UTC" deadline="2028-01-29 04:44:07.546834172 +0000 UTC" Jul 08 23:03:00.702535 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.702418 2548 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13661h41m6.844421131s" Jul 08 23:03:00.826339 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.826302 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:00.826524 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:00.826472 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:00.843644 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.843588 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-blm7c" event={"ID":"40709248-e1df-4adb-ac19-9be50f5cf61b","Type":"ContainerStarted","Data":"f538cddab3a75067d4bbfa232aaa20dc56578460ebeeb0a67afc9497cbbd66b8"} Jul 08 23:03:00.856873 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.856784 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" event={"ID":"608dffea-c3de-4379-9289-433428dc33f0","Type":"ContainerStarted","Data":"38660bf8426c3a223a0f7958ed6db19beab55bc86bf87fd0470f6cf61d6f26aa"} Jul 08 23:03:00.869359 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.869264 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nwsqm" event={"ID":"58690598-bf03-4609-bc53-3d4b5edca7fc","Type":"ContainerStarted","Data":"a160b80cddad9a0a9390cf01dcd2aea2f39dcf1fd78e3ffc0b50c451bf8e896f"} Jul 08 23:03:00.875263 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.875226 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x6gwk" event={"ID":"aa3acccd-8292-46b0-b34b-d6ecba8fc595","Type":"ContainerStarted","Data":"8a942218af1de610e17374d9c593f9103c8c9c1a5c9dd47411017cca5da400cf"} Jul 08 23:03:00.900764 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.900721 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mt56m" event={"ID":"44817e25-f0f3-44fc-ab14-f1cc0cddd89e","Type":"ContainerStarted","Data":"43bd494ff052c6295506142a59f4cf92040e9bf3bd0d41f3e48ed3fed840bd11"} Jul 08 23:03:00.920695 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.917653 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wnnrr" event={"ID":"c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28","Type":"ContainerStarted","Data":"dea5ef98d2a49b3aaaa6335425258bce8aa2c112eb2ecb377baf276d9222273a"} Jul 08 23:03:00.950618 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.950551 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jv2x8" event={"ID":"3345846f-189a-4f26-bd90-737c47f0f938","Type":"ContainerStarted","Data":"94eee795992b8c9a3f22d072e06e7a9fd2e8be2d7145e8aa1d76a750de933c28"} Jul 08 23:03:00.961917 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.961780 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mdhnl" event={"ID":"9984b179-b525-4ea8-b339-be8b23fc35b2","Type":"ContainerStarted","Data":"aaab9548586d5e5493fb798f8daa358f97b593527c25cd9872e52df4d499a42e"} Jul 08 23:03:00.974985 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:00.974793 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" event={"ID":"b5236fd2-5d92-4b80-afe4-68b9bbb567b8","Type":"ContainerStarted","Data":"78749f2f5916d2b4a008545c5ec42611c7ed3343c155be7920881aee3d04673e"} Jul 08 23:03:01.384002 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:01.383061 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs\") pod \"network-metrics-daemon-4qwwh\" (UID: \"48abd27d-5a99-4c70-937b-b9d06f033843\") " pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:01.384002 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:01.383122 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret\") pod \"global-pull-secret-syncer-dwldq\" (UID: \"213525b4-d967-437a-9c90-d40f2190a5a5\") " pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:01.384002 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:01.383307 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 23:03:01.384002 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:01.383377 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret podName:213525b4-d967-437a-9c90-d40f2190a5a5 nodeName:}" failed. No retries permitted until 2026-07-08 23:03:03.383357351 +0000 UTC m=+5.143852402 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret") pod "global-pull-secret-syncer-dwldq" (UID: "213525b4-d967-437a-9c90-d40f2190a5a5") : object "kube-system"/"original-pull-secret" not registered Jul 08 23:03:01.384002 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:01.383847 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 23:03:01.384002 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:01.383900 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs podName:48abd27d-5a99-4c70-937b-b9d06f033843 nodeName:}" failed. No retries permitted until 2026-07-08 23:03:03.383883109 +0000 UTC m=+5.144378150 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs") pod "network-metrics-daemon-4qwwh" (UID: "48abd27d-5a99-4c70-937b-b9d06f033843") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 23:03:01.483946 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:01.483904 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t67kc\" (UniqueName: \"kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc\") pod \"network-check-target-chcsc\" (UID: \"825c51de-1ee6-48cd-8e9c-66811e605fbf\") " pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:01.484141 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:01.484109 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 23:03:01.484141 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:01.484135 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 23:03:01.484266 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:01.484149 2548 projected.go:196] Error preparing data for projected volume kube-api-access-t67kc for pod openshift-network-diagnostics/network-check-target-chcsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 23:03:01.484266 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:01.484213 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc podName:825c51de-1ee6-48cd-8e9c-66811e605fbf nodeName:}" failed. No retries permitted until 2026-07-08 23:03:03.484193701 +0000 UTC m=+5.244688753 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-t67kc" (UniqueName: "kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc") pod "network-check-target-chcsc" (UID: "825c51de-1ee6-48cd-8e9c-66811e605fbf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 23:03:01.703732 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:01.703574 2548 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 22:58:00 +0000 UTC" deadline="2028-03-14 03:20:30.427363246 +0000 UTC" Jul 08 23:03:01.703732 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:01.703615 2548 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14740h17m28.723753323s" Jul 08 23:03:01.823571 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:01.823534 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:01.823789 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:01.823765 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:01.824452 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:01.823861 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:01.824452 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:01.824265 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:02.077201 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:02.077082 2548 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 23:03:02.825972 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:02.825611 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:02.826498 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:02.826011 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:03.401959 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:03.401080 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret\") pod \"global-pull-secret-syncer-dwldq\" (UID: \"213525b4-d967-437a-9c90-d40f2190a5a5\") " pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:03.401959 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:03.401207 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs\") pod \"network-metrics-daemon-4qwwh\" (UID: \"48abd27d-5a99-4c70-937b-b9d06f033843\") " pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:03.401959 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:03.401351 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 23:03:03.401959 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:03.401417 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs podName:48abd27d-5a99-4c70-937b-b9d06f033843 nodeName:}" failed. No retries permitted until 2026-07-08 23:03:07.401396767 +0000 UTC m=+9.161891825 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs") pod "network-metrics-daemon-4qwwh" (UID: "48abd27d-5a99-4c70-937b-b9d06f033843") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 23:03:03.401959 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:03.401859 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 23:03:03.401959 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:03.401914 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret podName:213525b4-d967-437a-9c90-d40f2190a5a5 nodeName:}" failed. No retries permitted until 2026-07-08 23:03:07.401896123 +0000 UTC m=+9.162391157 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret") pod "global-pull-secret-syncer-dwldq" (UID: "213525b4-d967-437a-9c90-d40f2190a5a5") : object "kube-system"/"original-pull-secret" not registered Jul 08 23:03:03.502133 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:03.502042 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t67kc\" (UniqueName: \"kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc\") pod \"network-check-target-chcsc\" (UID: \"825c51de-1ee6-48cd-8e9c-66811e605fbf\") " pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:03.502406 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:03.502314 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 23:03:03.502406 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:03.502335 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 23:03:03.502406 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:03.502349 2548 projected.go:196] Error preparing data for projected volume kube-api-access-t67kc for pod openshift-network-diagnostics/network-check-target-chcsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 23:03:03.502589 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:03.502412 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc podName:825c51de-1ee6-48cd-8e9c-66811e605fbf nodeName:}" failed. No retries permitted until 2026-07-08 23:03:07.502392318 +0000 UTC m=+9.262887369 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-t67kc" (UniqueName: "kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc") pod "network-check-target-chcsc" (UID: "825c51de-1ee6-48cd-8e9c-66811e605fbf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 23:03:03.823233 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:03.823147 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:03.823398 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:03.823156 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:03.823398 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:03.823305 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:03.823522 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:03.823394 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:04.823118 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:04.823080 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:04.823609 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:04.823230 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:05.822933 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:05.822880 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:05.823138 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:05.823033 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:05.823524 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:05.822900 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:05.823524 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:05.823446 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:06.823036 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:06.822939 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:06.823214 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:06.823077 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:07.435545 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:07.435503 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs\") pod \"network-metrics-daemon-4qwwh\" (UID: \"48abd27d-5a99-4c70-937b-b9d06f033843\") " pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:07.435769 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:07.435560 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret\") pod \"global-pull-secret-syncer-dwldq\" (UID: \"213525b4-d967-437a-9c90-d40f2190a5a5\") " pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:07.435769 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:07.435667 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 23:03:07.435769 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:07.435718 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 23:03:07.435769 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:07.435750 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs podName:48abd27d-5a99-4c70-937b-b9d06f033843 nodeName:}" failed. No retries permitted until 2026-07-08 23:03:15.435726786 +0000 UTC m=+17.196221835 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs") pod "network-metrics-daemon-4qwwh" (UID: "48abd27d-5a99-4c70-937b-b9d06f033843") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 23:03:07.436014 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:07.435787 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret podName:213525b4-d967-437a-9c90-d40f2190a5a5 nodeName:}" failed. No retries permitted until 2026-07-08 23:03:15.435765193 +0000 UTC m=+17.196260266 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret") pod "global-pull-secret-syncer-dwldq" (UID: "213525b4-d967-437a-9c90-d40f2190a5a5") : object "kube-system"/"original-pull-secret" not registered Jul 08 23:03:07.536835 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:07.536795 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t67kc\" (UniqueName: \"kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc\") pod \"network-check-target-chcsc\" (UID: \"825c51de-1ee6-48cd-8e9c-66811e605fbf\") " pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:07.537031 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:07.536991 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 23:03:07.537031 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:07.537017 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 23:03:07.537031 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:07.537031 2548 projected.go:196] Error preparing data for projected volume kube-api-access-t67kc for pod openshift-network-diagnostics/network-check-target-chcsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 23:03:07.537204 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:07.537107 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc podName:825c51de-1ee6-48cd-8e9c-66811e605fbf nodeName:}" failed. No retries permitted until 2026-07-08 23:03:15.537084292 +0000 UTC m=+17.297579347 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-t67kc" (UniqueName: "kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc") pod "network-check-target-chcsc" (UID: "825c51de-1ee6-48cd-8e9c-66811e605fbf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 23:03:07.823761 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:07.823684 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:07.823761 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:07.823705 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:07.824237 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:07.823844 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:07.824237 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:07.824009 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:08.823894 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:08.823713 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:08.823894 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:08.823816 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:09.823636 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:09.823552 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:09.823836 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:09.823711 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:09.823964 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:09.823939 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:09.824352 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:09.824068 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:10.823437 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:10.823403 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:10.823653 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:10.823539 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:11.823580 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:11.823543 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:11.824045 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:11.823591 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:11.824045 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:11.823701 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:11.824045 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:11.823848 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:12.823782 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:12.823747 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:12.824287 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:12.823885 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:13.823710 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:13.823670 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:13.823893 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:13.823683 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:13.823893 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:13.823790 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:13.824289 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:13.823886 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:14.823360 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:14.823324 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:14.823527 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:14.823456 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:15.492550 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:15.492510 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs\") pod \"network-metrics-daemon-4qwwh\" (UID: \"48abd27d-5a99-4c70-937b-b9d06f033843\") " pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:15.493051 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:15.492565 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret\") pod \"global-pull-secret-syncer-dwldq\" (UID: \"213525b4-d967-437a-9c90-d40f2190a5a5\") " pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:15.493051 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:15.492687 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 23:03:15.493051 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:15.492741 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs podName:48abd27d-5a99-4c70-937b-b9d06f033843 nodeName:}" failed. No retries permitted until 2026-07-08 23:03:31.492727477 +0000 UTC m=+33.253222509 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs") pod "network-metrics-daemon-4qwwh" (UID: "48abd27d-5a99-4c70-937b-b9d06f033843") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 23:03:15.493051 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:15.492837 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 23:03:15.493051 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:15.492904 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret podName:213525b4-d967-437a-9c90-d40f2190a5a5 nodeName:}" failed. No retries permitted until 2026-07-08 23:03:31.49288795 +0000 UTC m=+33.253382982 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret") pod "global-pull-secret-syncer-dwldq" (UID: "213525b4-d967-437a-9c90-d40f2190a5a5") : object "kube-system"/"original-pull-secret" not registered Jul 08 23:03:15.593496 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:15.593458 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t67kc\" (UniqueName: \"kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc\") pod \"network-check-target-chcsc\" (UID: \"825c51de-1ee6-48cd-8e9c-66811e605fbf\") " pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:15.593696 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:15.593615 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 23:03:15.593696 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:15.593643 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 23:03:15.593696 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:15.593655 2548 projected.go:196] Error preparing data for projected volume kube-api-access-t67kc for pod openshift-network-diagnostics/network-check-target-chcsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 23:03:15.593856 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:15.593718 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc podName:825c51de-1ee6-48cd-8e9c-66811e605fbf nodeName:}" failed. No retries permitted until 2026-07-08 23:03:31.59370014 +0000 UTC m=+33.354195172 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-t67kc" (UniqueName: "kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc") pod "network-check-target-chcsc" (UID: "825c51de-1ee6-48cd-8e9c-66811e605fbf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 23:03:15.823210 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:15.823129 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:15.823210 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:15.823159 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:15.823381 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:15.823252 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:15.823381 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:15.823366 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:16.823166 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:16.823128 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:16.823643 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:16.823269 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:17.823400 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:17.823365 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:17.823887 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:17.823365 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:17.823887 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:17.823497 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:17.823887 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:17.823623 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:18.823787 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:18.823736 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:18.824271 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:18.823852 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:19.823298 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:19.823257 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:19.823489 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:19.823312 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:19.823489 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:19.823453 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:19.823645 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:19.823590 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:20.823477 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:20.822954 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:20.824219 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:20.823503 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:21.025767 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.025730 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-blm7c" event={"ID":"40709248-e1df-4adb-ac19-9be50f5cf61b","Type":"ContainerStarted","Data":"e6f0ff187514953cf12d5284e941871cf8f744efab82db6c0711efe44abc342c"} Jul 08 23:03:21.027147 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.027117 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" event={"ID":"608dffea-c3de-4379-9289-433428dc33f0","Type":"ContainerStarted","Data":"1dbf9352d17dbf4929e2483571d4352fd057def049716d885211bcbd22024e82"} Jul 08 23:03:21.028413 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.028388 2548 generic.go:356] "Generic (PLEG): container finished" podID="58690598-bf03-4609-bc53-3d4b5edca7fc" containerID="95050d8cb377a673852f5a804da1395737071ed9d8674585117ea61c8a77e30a" exitCode=0 Jul 08 23:03:21.028523 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.028460 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nwsqm" event={"ID":"58690598-bf03-4609-bc53-3d4b5edca7fc","Type":"ContainerDied","Data":"95050d8cb377a673852f5a804da1395737071ed9d8674585117ea61c8a77e30a"} Jul 08 23:03:21.030029 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.030007 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x6gwk" event={"ID":"aa3acccd-8292-46b0-b34b-d6ecba8fc595","Type":"ContainerStarted","Data":"50baab7d1621ae3651ed7ccd74568aa858af1b98f1edbe3a4befe267a51733f6"} Jul 08 23:03:21.031440 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.031422 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mt56m" event={"ID":"44817e25-f0f3-44fc-ab14-f1cc0cddd89e","Type":"ContainerStarted","Data":"c880565c8092f5f5e913137b5de44b9d0c15ab96b12ff29675c365ac0404cd9b"} Jul 08 23:03:21.032768 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.032744 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wnnrr" event={"ID":"c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28","Type":"ContainerStarted","Data":"b0496ea9431fe81a6e179aec93eb5a8e844dbfea0b211e36e65806a12de842df"} Jul 08 23:03:21.034128 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.034107 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mdhnl" event={"ID":"9984b179-b525-4ea8-b339-be8b23fc35b2","Type":"ContainerStarted","Data":"788a715efde4d3325e644960669127f39d185f9b736855c34e2e059a810f0577"} Jul 08 23:03:21.037233 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.037213 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" event={"ID":"b5236fd2-5d92-4b80-afe4-68b9bbb567b8","Type":"ContainerStarted","Data":"ae59b6b83458df4fbedb896bbeef85d02e177679b95e9f3fab7fe58bb4f4a133"} Jul 08 23:03:21.037330 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.037268 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" event={"ID":"b5236fd2-5d92-4b80-afe4-68b9bbb567b8","Type":"ContainerStarted","Data":"e0153aa5e9e9eb52f8fe13fdfa4c9b9031d0c64d352d1a60392d2ffac8ceac3c"} Jul 08 23:03:21.037330 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.037300 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" event={"ID":"b5236fd2-5d92-4b80-afe4-68b9bbb567b8","Type":"ContainerStarted","Data":"af1a9c010f9e88dcf86cbf3888c4c6049c6afa47cb339f4e4b767a95064d2b10"} Jul 08 23:03:21.037428 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.037336 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" event={"ID":"b5236fd2-5d92-4b80-afe4-68b9bbb567b8","Type":"ContainerStarted","Data":"8d22a8f1201f8c88fedda6ef520a7bcffb651520787fa2bae7e62a7bd7498cf6"} Jul 08 23:03:21.037428 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.037363 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" event={"ID":"b5236fd2-5d92-4b80-afe4-68b9bbb567b8","Type":"ContainerStarted","Data":"792ce1c54ccddfadbf29b44a1670294f2a014eec17e223153b05799df7e59d21"} Jul 08 23:03:21.037498 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.037437 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" event={"ID":"b5236fd2-5d92-4b80-afe4-68b9bbb567b8","Type":"ContainerStarted","Data":"c75a91a9b805a7585d47fc1ed8b346c55073a6b41fa8b8ec7ed95159c35cc3c9"} Jul 08 23:03:21.038647 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.038629 2548 generic.go:356] "Generic (PLEG): container finished" podID="641c0d2e09eac1bc2f01a3f9dd96a166" containerID="fef7ca283a3c4c0f79da489a2fd96bf777b41cfa658c2632cb97d370309d0429" exitCode=0 Jul 08 23:03:21.038717 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.038687 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal" event={"ID":"641c0d2e09eac1bc2f01a3f9dd96a166","Type":"ContainerDied","Data":"fef7ca283a3c4c0f79da489a2fd96bf777b41cfa658c2632cb97d370309d0429"} Jul 08 23:03:21.040147 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.040128 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-106.ec2.internal" event={"ID":"c95e074b8f38074e93586ba30700e2a9","Type":"ContainerStarted","Data":"f173e57135af931e2d87a1d9bccda2fe2d08d36b7e1835086ab9e548d4459556"} Jul 08 23:03:21.042110 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.042073 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-blm7c" podStartSLOduration=2.065020892 podStartE2EDuration="22.042056979s" totalImagesPullingTime="19.977036087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:02:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:21.041769304 +0000 UTC m=+22.802264370" watchObservedRunningTime="2026-07-08 23:03:21.042056979 +0000 UTC m=+22.802552037" Jul 08 23:03:21.054771 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.054735 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wnnrr" podStartSLOduration=1.97626184 podStartE2EDuration="22.054723975s" totalImagesPullingTime="20.078462135s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:02:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:21.054407727 +0000 UTC m=+22.814902793" watchObservedRunningTime="2026-07-08 23:03:21.054723975 +0000 UTC m=+22.815219028" Jul 08 23:03:21.083152 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.083092 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mdhnl" podStartSLOduration=2.015899074 podStartE2EDuration="22.083069961s" totalImagesPullingTime="20.067170887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:02:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:21.069521816 +0000 UTC m=+22.830016891" watchObservedRunningTime="2026-07-08 23:03:21.083069961 +0000 UTC m=+22.843565017" Jul 08 23:03:21.094887 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.094830 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-106.ec2.internal" podStartSLOduration=21.094808034 podStartE2EDuration="21.094808034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:03:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:21.094727227 +0000 UTC m=+22.855222282" watchObservedRunningTime="2026-07-08 23:03:21.094808034 +0000 UTC m=+22.855303090" Jul 08 23:03:21.111404 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.111359 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mt56m" podStartSLOduration=2.054006127 podStartE2EDuration="22.111344391s" totalImagesPullingTime="20.057338264s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:02:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:21.110617346 +0000 UTC m=+22.871112391" watchObservedRunningTime="2026-07-08 23:03:21.111344391 +0000 UTC m=+22.871839446" Jul 08 23:03:21.124710 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.124631 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-x6gwk" podStartSLOduration=2.085792315 podStartE2EDuration="22.124618596s" totalImagesPullingTime="20.038826281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:02:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:21.124060666 +0000 UTC m=+22.884555720" watchObservedRunningTime="2026-07-08 23:03:21.124618596 +0000 UTC m=+22.885113684" Jul 08 23:03:21.823372 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.823340 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:21.823663 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:21.823475 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:21.823663 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:21.823531 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:21.824063 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:21.823681 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:22.043546 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:22.043507 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jv2x8" event={"ID":"3345846f-189a-4f26-bd90-737c47f0f938","Type":"ContainerStarted","Data":"03c987e0d85dff24f65165068c448a29e28c0138ed80c274f7de1c4e773caf53"} Jul 08 23:03:22.045540 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:22.045514 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal" event={"ID":"641c0d2e09eac1bc2f01a3f9dd96a166","Type":"ContainerStarted","Data":"42738b616706ed53d4424d1a278424b927acaee686eb06f1c5c40b5c483e38c1"} Jul 08 23:03:22.059643 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:22.059571 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jv2x8" podStartSLOduration=3.051714107 podStartE2EDuration="23.059555013s" totalImagesPullingTime="20.007840906s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:02:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:22.058980787 +0000 UTC m=+23.819475840" watchObservedRunningTime="2026-07-08 23:03:22.059555013 +0000 UTC m=+23.820050067" Jul 08 23:03:22.073322 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:22.073281 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-106.ec2.internal" podStartSLOduration=22.073268867 podStartE2EDuration="22.073268867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:03:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:22.073183072 +0000 UTC m=+23.833678130" watchObservedRunningTime="2026-07-08 23:03:22.073268867 +0000 UTC m=+23.833763921" Jul 08 23:03:22.220251 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:22.220226 2548 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 23:03:22.748928 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:22.748811 2548 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T23:03:22.220247112Z","UUID":"6156c17f-1bfd-4eec-8825-26266a94d47d","Handler":null,"Name":"","Endpoint":""} Jul 08 23:03:22.750333 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:22.750310 2548 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 23:03:22.750453 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:22.750363 2548 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 23:03:22.823738 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:22.823708 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:22.824195 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:22.823878 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:23.050387 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:23.050290 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" event={"ID":"608dffea-c3de-4379-9289-433428dc33f0","Type":"ContainerStarted","Data":"4f3d0518778f9fb78f3b95918f5dfdd66f75a98088a53826ceee611e320f3c9b"} Jul 08 23:03:23.054810 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:23.054774 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" event={"ID":"b5236fd2-5d92-4b80-afe4-68b9bbb567b8","Type":"ContainerStarted","Data":"1149bf987d57d015bf0a553abaa50eb71f1866cb7ce538b9cbd25b36e4e557d3"} Jul 08 23:03:23.088907 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:23.088880 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wnnrr_c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28/dns-node-resolver/0.log" Jul 08 23:03:23.823114 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:23.823025 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:23.823114 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:23.823077 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:23.823297 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:23.823147 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:23.823297 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:23.823258 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:23.867099 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:23.866919 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mt56m_44817e25-f0f3-44fc-ab14-f1cc0cddd89e/node-ca/0.log" Jul 08 23:03:24.059962 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:24.059926 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" event={"ID":"608dffea-c3de-4379-9289-433428dc33f0","Type":"ContainerStarted","Data":"711d8d08f70441d860405dbfc3ea618d02ef0cd0fa00e783e75819f9ec6a550a"} Jul 08 23:03:24.078043 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:24.077953 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r76jt" podStartSLOduration=2.539810322 podStartE2EDuration="25.077938659s" totalImagesPullingTime="22.538128337s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:02:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:24.076689542 +0000 UTC m=+25.837184623" watchObservedRunningTime="2026-07-08 23:03:24.077938659 +0000 UTC m=+25.838433713" Jul 08 23:03:24.822997 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:24.822968 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:24.823170 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:24.823066 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:24.967713 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:24.967679 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-x6gwk" Jul 08 23:03:24.968414 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:24.968382 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-x6gwk" Jul 08 23:03:25.092647 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:25.092492 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-x6gwk" Jul 08 23:03:25.093011 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:25.092990 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x6gwk" Jul 08 23:03:25.823740 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:25.823710 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:25.823898 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:25.823710 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:25.823898 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:25.823813 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:25.823964 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:25.823893 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:26.070453 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:26.070378 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" event={"ID":"b5236fd2-5d92-4b80-afe4-68b9bbb567b8","Type":"ContainerStarted","Data":"8007655dbc8417ec5b3f26bb2eedeac84ba64213c893c83b950c8627db8df776"} Jul 08 23:03:26.071213 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:26.070696 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:03:26.071213 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:26.070777 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:03:26.087328 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:26.087254 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:03:26.106635 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:26.106565 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" podStartSLOduration=7.002373421 podStartE2EDuration="27.106551999s" totalImagesPullingTime="20.104178578s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:02:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:26.10497031 +0000 UTC m=+27.865465375" watchObservedRunningTime="2026-07-08 23:03:26.106551999 +0000 UTC m=+27.867047052" Jul 08 23:03:26.823461 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:26.823220 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:26.823623 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:26.823495 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:27.073208 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:27.073176 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:03:27.090379 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:27.090303 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:03:27.465482 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:27.465404 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-chcsc"] Jul 08 23:03:27.465652 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:27.465511 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:27.465652 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:27.465628 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:27.467351 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:27.467323 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dwldq"] Jul 08 23:03:27.467468 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:27.467451 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:27.467582 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:27.467560 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:27.468060 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:27.468036 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4qwwh"] Jul 08 23:03:27.468165 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:27.468145 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:27.468285 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:27.468257 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:28.824525 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:28.824485 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:28.825064 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:28.824575 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:28.825064 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:28.824622 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:28.825064 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:28.824713 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:29.078884 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:29.078801 2548 generic.go:356] "Generic (PLEG): container finished" podID="58690598-bf03-4609-bc53-3d4b5edca7fc" containerID="1ef6a6abffbc81edea0cde4ad3b87d32b3bff0f88d50e6f7280ae5dd8a51a3b2" exitCode=0 Jul 08 23:03:29.079028 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:29.078893 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nwsqm" event={"ID":"58690598-bf03-4609-bc53-3d4b5edca7fc","Type":"ContainerDied","Data":"1ef6a6abffbc81edea0cde4ad3b87d32b3bff0f88d50e6f7280ae5dd8a51a3b2"} Jul 08 23:03:29.823727 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:29.823635 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:29.823857 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:29.823784 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:30.082744 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:30.082427 2548 generic.go:356] "Generic (PLEG): container finished" podID="58690598-bf03-4609-bc53-3d4b5edca7fc" containerID="cb8cb582ed0d628937f36281f348c4b3b84908b3ed6271d0663aaf6639212f88" exitCode=0 Jul 08 23:03:30.082744 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:30.082505 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nwsqm" event={"ID":"58690598-bf03-4609-bc53-3d4b5edca7fc","Type":"ContainerDied","Data":"cb8cb582ed0d628937f36281f348c4b3b84908b3ed6271d0663aaf6639212f88"} Jul 08 23:03:30.823679 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:30.823639 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:30.823845 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:30.823759 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:30.823845 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:30.823816 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:30.823947 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:30.823929 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:31.087545 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:31.087514 2548 generic.go:356] "Generic (PLEG): container finished" podID="58690598-bf03-4609-bc53-3d4b5edca7fc" containerID="e99b30cc10b75bcee857b0b0cccc1cee69ec4e74a15c2571fec3d0704c12b872" exitCode=0 Jul 08 23:03:31.087909 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:31.087569 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nwsqm" event={"ID":"58690598-bf03-4609-bc53-3d4b5edca7fc","Type":"ContainerDied","Data":"e99b30cc10b75bcee857b0b0cccc1cee69ec4e74a15c2571fec3d0704c12b872"} Jul 08 23:03:31.521408 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:31.521375 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs\") pod \"network-metrics-daemon-4qwwh\" (UID: \"48abd27d-5a99-4c70-937b-b9d06f033843\") " pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:31.521408 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:31.521410 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret\") pod \"global-pull-secret-syncer-dwldq\" (UID: \"213525b4-d967-437a-9c90-d40f2190a5a5\") " pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:31.521628 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:31.521516 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 23:03:31.521628 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:31.521521 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 23:03:31.521628 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:31.521566 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret podName:213525b4-d967-437a-9c90-d40f2190a5a5 nodeName:}" failed. No retries permitted until 2026-07-08 23:04:03.521552826 +0000 UTC m=+65.282047857 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret") pod "global-pull-secret-syncer-dwldq" (UID: "213525b4-d967-437a-9c90-d40f2190a5a5") : object "kube-system"/"original-pull-secret" not registered Jul 08 23:03:31.521628 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:31.521578 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs podName:48abd27d-5a99-4c70-937b-b9d06f033843 nodeName:}" failed. No retries permitted until 2026-07-08 23:04:03.521571894 +0000 UTC m=+65.282066926 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs") pod "network-metrics-daemon-4qwwh" (UID: "48abd27d-5a99-4c70-937b-b9d06f033843") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 23:03:31.622036 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:31.622006 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t67kc\" (UniqueName: \"kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc\") pod \"network-check-target-chcsc\" (UID: \"825c51de-1ee6-48cd-8e9c-66811e605fbf\") " pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:31.622221 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:31.622135 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 23:03:31.622221 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:31.622155 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 23:03:31.622221 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:31.622165 2548 projected.go:196] Error preparing data for projected volume kube-api-access-t67kc for pod openshift-network-diagnostics/network-check-target-chcsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 23:03:31.622373 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:31.622222 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc podName:825c51de-1ee6-48cd-8e9c-66811e605fbf nodeName:}" failed. No retries permitted until 2026-07-08 23:04:03.622203793 +0000 UTC m=+65.382698825 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-t67kc" (UniqueName: "kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc") pod "network-check-target-chcsc" (UID: "825c51de-1ee6-48cd-8e9c-66811e605fbf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 23:03:31.823461 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:31.823379 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:31.823629 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:31.823486 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:32.825743 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:32.825715 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:32.826140 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:32.825715 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:32.826140 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:32.825808 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:32.826140 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:32.825885 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:33.823793 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:33.823758 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:33.824037 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:33.823871 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:34.826714 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:34.826687 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:34.827109 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:34.826793 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:34.827109 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:34.826847 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:34.827109 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:34.826960 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:35.823183 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:35.823150 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:35.823365 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:35.823292 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:36.822991 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:36.822954 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:36.823573 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:36.823065 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:36.823573 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:36.823094 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:36.823573 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:36.823153 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:37.103250 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:37.103177 2548 generic.go:356] "Generic (PLEG): container finished" podID="58690598-bf03-4609-bc53-3d4b5edca7fc" containerID="d95760c3dfc120ead63b0f02d0f184ba210d8105f722d7754c0cf909e5f4fd42" exitCode=0 Jul 08 23:03:37.103250 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:37.103218 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nwsqm" event={"ID":"58690598-bf03-4609-bc53-3d4b5edca7fc","Type":"ContainerDied","Data":"d95760c3dfc120ead63b0f02d0f184ba210d8105f722d7754c0cf909e5f4fd42"} Jul 08 23:03:37.823732 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:37.823701 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:37.824275 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:37.823833 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:38.108347 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:38.108275 2548 generic.go:356] "Generic (PLEG): container finished" podID="58690598-bf03-4609-bc53-3d4b5edca7fc" containerID="bac1a2aa9501c915a15e8287e6cf3e120bb3dfac495a5b9efbb3b7842901c870" exitCode=0 Jul 08 23:03:38.108347 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:38.108328 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nwsqm" event={"ID":"58690598-bf03-4609-bc53-3d4b5edca7fc","Type":"ContainerDied","Data":"bac1a2aa9501c915a15e8287e6cf3e120bb3dfac495a5b9efbb3b7842901c870"} Jul 08 23:03:38.824227 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:38.824190 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:38.824824 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:38.824295 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:38.824824 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:38.824349 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:38.824824 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:38.824435 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:39.113693 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:39.113590 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nwsqm" event={"ID":"58690598-bf03-4609-bc53-3d4b5edca7fc","Type":"ContainerStarted","Data":"923542156d9bc598e1f894e2230d2601fdccff749efa9afdd04bb7bc776bc6f3"} Jul 08 23:03:39.136429 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:39.136358 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nwsqm" podStartSLOduration=6.197384703 podStartE2EDuration="40.136339878s" totalImagesPullingTime="33.938955175s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:02:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:39.136199391 +0000 UTC m=+40.896694470" watchObservedRunningTime="2026-07-08 23:03:39.136339878 +0000 UTC m=+40.896834933" Jul 08 23:03:39.822938 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:39.822907 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:39.823108 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:39.823014 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:40.825611 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:40.825574 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:40.825973 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:40.825627 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:40.825973 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:40.825703 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:40.825973 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:40.825808 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:41.823761 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:41.823731 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:41.823931 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:41.823841 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:42.825691 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:42.825661 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:42.826051 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:42.825759 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:42.826051 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:42.825809 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:42.826051 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:42.825885 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:43.823050 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:43.823016 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:43.823218 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:43.823128 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:44.823343 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:44.823316 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:44.823760 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:44.823416 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:44.823760 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:44.823477 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:44.823760 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:44.823585 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:45.823428 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:45.823399 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:45.823820 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:45.823503 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:46.822757 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:46.822731 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:46.822931 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:46.822740 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:46.822931 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:46.822836 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:46.823033 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:46.822927 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:47.823613 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:47.823578 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:47.823959 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:47.823719 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qwwh" podUID="48abd27d-5a99-4c70-937b-b9d06f033843" Jul 08 23:03:48.824160 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:48.824087 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:48.824521 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:48.824169 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chcsc" podUID="825c51de-1ee6-48cd-8e9c-66811e605fbf" Jul 08 23:03:48.824521 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:48.824270 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:48.824521 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:03:48.824397 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dwldq" podUID="213525b4-d967-437a-9c90-d40f2190a5a5" Jul 08 23:03:49.059172 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.059147 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-106.ec2.internal" event="NodeReady" Jul 08 23:03:49.059362 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.059266 2548 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 23:03:49.097623 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.097578 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7cc7b69d54-6vkwp"] Jul 08 23:03:49.108609 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.108579 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.111813 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.111792 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 23:03:49.111925 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.111795 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qxxgd\"" Jul 08 23:03:49.112033 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.112015 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 23:03:49.112428 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.112404 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 23:03:49.112909 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.112888 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cc7b69d54-6vkwp"] Jul 08 23:03:49.116827 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.116807 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 23:03:49.139241 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.139209 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hsjss"] Jul 08 23:03:49.149727 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.149694 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-x4nvj"] Jul 08 23:03:49.149899 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.149812 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hsjss" Jul 08 23:03:49.153989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.153237 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 23:03:49.153989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.153515 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 23:03:49.154436 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.154420 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xlh5l\"" Jul 08 23:03:49.156012 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.155990 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1fc2cc5c-7632-4171-9813-d13c9b038424-registry-tls\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.156105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.156024 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1fc2cc5c-7632-4171-9813-d13c9b038424-registry-certificates\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.156105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.156043 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1fc2cc5c-7632-4171-9813-d13c9b038424-installation-pull-secrets\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.156105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.156062 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1fc2cc5c-7632-4171-9813-d13c9b038424-bound-sa-token\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.156105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.156084 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld8br\" (UniqueName: \"kubernetes.io/projected/1fc2cc5c-7632-4171-9813-d13c9b038424-kube-api-access-ld8br\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.156270 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.156122 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8fae214-eb17-4ee8-b4d2-de209a11ad64-cert\") pod \"ingress-canary-hsjss\" (UID: \"b8fae214-eb17-4ee8-b4d2-de209a11ad64\") " pod="openshift-ingress-canary/ingress-canary-hsjss" Jul 08 23:03:49.156270 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.156175 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzwbl\" (UniqueName: \"kubernetes.io/projected/b8fae214-eb17-4ee8-b4d2-de209a11ad64-kube-api-access-xzwbl\") pod \"ingress-canary-hsjss\" (UID: \"b8fae214-eb17-4ee8-b4d2-de209a11ad64\") " pod="openshift-ingress-canary/ingress-canary-hsjss" Jul 08 23:03:49.156270 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.156197 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1fc2cc5c-7632-4171-9813-d13c9b038424-ca-trust-extracted\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.156270 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.156237 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1fc2cc5c-7632-4171-9813-d13c9b038424-trusted-ca\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.156442 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.156322 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1fc2cc5c-7632-4171-9813-d13c9b038424-image-registry-private-configuration\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.160240 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.160222 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x4nvj" Jul 08 23:03:49.162975 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.162946 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 23:03:49.163104 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.163077 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 23:03:49.163188 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.163119 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 23:03:49.163235 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.163198 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ntpx9\"" Jul 08 23:03:49.163504 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.163489 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 23:03:49.164274 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.164260 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 23:03:49.166417 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.166392 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hsjss"] Jul 08 23:03:49.185139 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.185106 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x4nvj"] Jul 08 23:03:49.258665 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.258630 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1fc2cc5c-7632-4171-9813-d13c9b038424-registry-tls\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.258665 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.258672 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1fc2cc5c-7632-4171-9813-d13c9b038424-registry-certificates\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.258890 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.258695 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1fc2cc5c-7632-4171-9813-d13c9b038424-installation-pull-secrets\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.258890 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.258722 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1fc2cc5c-7632-4171-9813-d13c9b038424-bound-sa-token\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.258890 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.258750 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/577bedd6-568d-4e24-ab8e-0ed9ae106385-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x4nvj\" (UID: \"577bedd6-568d-4e24-ab8e-0ed9ae106385\") " pod="openshift-insights/insights-runtime-extractor-x4nvj" Jul 08 23:03:49.258890 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.258778 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld8br\" (UniqueName: \"kubernetes.io/projected/1fc2cc5c-7632-4171-9813-d13c9b038424-kube-api-access-ld8br\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.258890 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.258805 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8fae214-eb17-4ee8-b4d2-de209a11ad64-cert\") pod \"ingress-canary-hsjss\" (UID: \"b8fae214-eb17-4ee8-b4d2-de209a11ad64\") " pod="openshift-ingress-canary/ingress-canary-hsjss" Jul 08 23:03:49.258890 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.258830 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/577bedd6-568d-4e24-ab8e-0ed9ae106385-crio-socket\") pod \"insights-runtime-extractor-x4nvj\" (UID: \"577bedd6-568d-4e24-ab8e-0ed9ae106385\") " pod="openshift-insights/insights-runtime-extractor-x4nvj" Jul 08 23:03:49.258890 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.258868 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/577bedd6-568d-4e24-ab8e-0ed9ae106385-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x4nvj\" (UID: \"577bedd6-568d-4e24-ab8e-0ed9ae106385\") " pod="openshift-insights/insights-runtime-extractor-x4nvj" Jul 08 23:03:49.259234 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.258897 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4fwp\" (UniqueName: \"kubernetes.io/projected/577bedd6-568d-4e24-ab8e-0ed9ae106385-kube-api-access-k4fwp\") pod \"insights-runtime-extractor-x4nvj\" (UID: \"577bedd6-568d-4e24-ab8e-0ed9ae106385\") " pod="openshift-insights/insights-runtime-extractor-x4nvj" Jul 08 23:03:49.259234 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.258936 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzwbl\" (UniqueName: \"kubernetes.io/projected/b8fae214-eb17-4ee8-b4d2-de209a11ad64-kube-api-access-xzwbl\") pod \"ingress-canary-hsjss\" (UID: \"b8fae214-eb17-4ee8-b4d2-de209a11ad64\") " pod="openshift-ingress-canary/ingress-canary-hsjss" Jul 08 23:03:49.259234 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.258962 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1fc2cc5c-7632-4171-9813-d13c9b038424-ca-trust-extracted\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.259234 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.258985 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1fc2cc5c-7632-4171-9813-d13c9b038424-trusted-ca\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.259234 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.259060 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1fc2cc5c-7632-4171-9813-d13c9b038424-image-registry-private-configuration\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.259234 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.259101 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/577bedd6-568d-4e24-ab8e-0ed9ae106385-data-volume\") pod \"insights-runtime-extractor-x4nvj\" (UID: \"577bedd6-568d-4e24-ab8e-0ed9ae106385\") " pod="openshift-insights/insights-runtime-extractor-x4nvj" Jul 08 23:03:49.260508 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.259580 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1fc2cc5c-7632-4171-9813-d13c9b038424-registry-certificates\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.260508 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.259588 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jb42p"] Jul 08 23:03:49.260508 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.259932 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1fc2cc5c-7632-4171-9813-d13c9b038424-ca-trust-extracted\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.260779 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.260760 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1fc2cc5c-7632-4171-9813-d13c9b038424-trusted-ca\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.263894 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.263833 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8fae214-eb17-4ee8-b4d2-de209a11ad64-cert\") pod \"ingress-canary-hsjss\" (UID: \"b8fae214-eb17-4ee8-b4d2-de209a11ad64\") " pod="openshift-ingress-canary/ingress-canary-hsjss" Jul 08 23:03:49.263894 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.263851 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1fc2cc5c-7632-4171-9813-d13c9b038424-installation-pull-secrets\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.263894 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.263832 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1fc2cc5c-7632-4171-9813-d13c9b038424-image-registry-private-configuration\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.263894 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.263855 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1fc2cc5c-7632-4171-9813-d13c9b038424-registry-tls\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.268543 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.268464 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1fc2cc5c-7632-4171-9813-d13c9b038424-bound-sa-token\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.268910 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.268889 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld8br\" (UniqueName: \"kubernetes.io/projected/1fc2cc5c-7632-4171-9813-d13c9b038424-kube-api-access-ld8br\") pod \"image-registry-7cc7b69d54-6vkwp\" (UID: \"1fc2cc5c-7632-4171-9813-d13c9b038424\") " pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.268910 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.268901 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzwbl\" (UniqueName: \"kubernetes.io/projected/b8fae214-eb17-4ee8-b4d2-de209a11ad64-kube-api-access-xzwbl\") pod \"ingress-canary-hsjss\" (UID: \"b8fae214-eb17-4ee8-b4d2-de209a11ad64\") " pod="openshift-ingress-canary/ingress-canary-hsjss" Jul 08 23:03:49.280002 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.279982 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jb42p"] Jul 08 23:03:49.280094 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.280050 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jb42p" Jul 08 23:03:49.282739 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.282720 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 23:03:49.282840 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.282779 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2zvtt\"" Jul 08 23:03:49.282840 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.282779 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 23:03:49.359569 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.359502 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/577bedd6-568d-4e24-ab8e-0ed9ae106385-data-volume\") pod \"insights-runtime-extractor-x4nvj\" (UID: \"577bedd6-568d-4e24-ab8e-0ed9ae106385\") " pod="openshift-insights/insights-runtime-extractor-x4nvj" Jul 08 23:03:49.359569 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.359533 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9a58df3-ab7d-4b82-9e17-b7b140727a4b-metrics-tls\") pod \"dns-default-jb42p\" (UID: \"f9a58df3-ab7d-4b82-9e17-b7b140727a4b\") " pod="openshift-dns/dns-default-jb42p" Jul 08 23:03:49.359569 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.359552 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv8pl\" (UniqueName: \"kubernetes.io/projected/f9a58df3-ab7d-4b82-9e17-b7b140727a4b-kube-api-access-xv8pl\") pod \"dns-default-jb42p\" (UID: \"f9a58df3-ab7d-4b82-9e17-b7b140727a4b\") " pod="openshift-dns/dns-default-jb42p" Jul 08 23:03:49.359806 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.359580 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/577bedd6-568d-4e24-ab8e-0ed9ae106385-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x4nvj\" (UID: \"577bedd6-568d-4e24-ab8e-0ed9ae106385\") " pod="openshift-insights/insights-runtime-extractor-x4nvj" Jul 08 23:03:49.359806 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.359625 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/577bedd6-568d-4e24-ab8e-0ed9ae106385-crio-socket\") pod \"insights-runtime-extractor-x4nvj\" (UID: \"577bedd6-568d-4e24-ab8e-0ed9ae106385\") " pod="openshift-insights/insights-runtime-extractor-x4nvj" Jul 08 23:03:49.359806 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.359653 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/577bedd6-568d-4e24-ab8e-0ed9ae106385-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x4nvj\" (UID: \"577bedd6-568d-4e24-ab8e-0ed9ae106385\") " pod="openshift-insights/insights-runtime-extractor-x4nvj" Jul 08 23:03:49.359806 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.359676 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9a58df3-ab7d-4b82-9e17-b7b140727a4b-tmp-dir\") pod \"dns-default-jb42p\" (UID: \"f9a58df3-ab7d-4b82-9e17-b7b140727a4b\") " pod="openshift-dns/dns-default-jb42p" Jul 08 23:03:49.359806 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.359694 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4fwp\" (UniqueName: \"kubernetes.io/projected/577bedd6-568d-4e24-ab8e-0ed9ae106385-kube-api-access-k4fwp\") pod \"insights-runtime-extractor-x4nvj\" (UID: \"577bedd6-568d-4e24-ab8e-0ed9ae106385\") " pod="openshift-insights/insights-runtime-extractor-x4nvj" Jul 08 23:03:49.359806 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.359723 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f9a58df3-ab7d-4b82-9e17-b7b140727a4b-config-volume\") pod \"dns-default-jb42p\" (UID: \"f9a58df3-ab7d-4b82-9e17-b7b140727a4b\") " pod="openshift-dns/dns-default-jb42p" Jul 08 23:03:49.360083 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.359817 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/577bedd6-568d-4e24-ab8e-0ed9ae106385-crio-socket\") pod \"insights-runtime-extractor-x4nvj\" (UID: \"577bedd6-568d-4e24-ab8e-0ed9ae106385\") " pod="openshift-insights/insights-runtime-extractor-x4nvj" Jul 08 23:03:49.360083 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.359876 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/577bedd6-568d-4e24-ab8e-0ed9ae106385-data-volume\") pod \"insights-runtime-extractor-x4nvj\" (UID: \"577bedd6-568d-4e24-ab8e-0ed9ae106385\") " pod="openshift-insights/insights-runtime-extractor-x4nvj" Jul 08 23:03:49.360194 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.360104 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/577bedd6-568d-4e24-ab8e-0ed9ae106385-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x4nvj\" (UID: \"577bedd6-568d-4e24-ab8e-0ed9ae106385\") " pod="openshift-insights/insights-runtime-extractor-x4nvj" Jul 08 23:03:49.362030 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.362011 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/577bedd6-568d-4e24-ab8e-0ed9ae106385-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x4nvj\" (UID: \"577bedd6-568d-4e24-ab8e-0ed9ae106385\") " pod="openshift-insights/insights-runtime-extractor-x4nvj" Jul 08 23:03:49.367725 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.367704 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4fwp\" (UniqueName: \"kubernetes.io/projected/577bedd6-568d-4e24-ab8e-0ed9ae106385-kube-api-access-k4fwp\") pod \"insights-runtime-extractor-x4nvj\" (UID: \"577bedd6-568d-4e24-ab8e-0ed9ae106385\") " pod="openshift-insights/insights-runtime-extractor-x4nvj" Jul 08 23:03:49.419736 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.419702 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:49.460120 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.460087 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9a58df3-ab7d-4b82-9e17-b7b140727a4b-tmp-dir\") pod \"dns-default-jb42p\" (UID: \"f9a58df3-ab7d-4b82-9e17-b7b140727a4b\") " pod="openshift-dns/dns-default-jb42p" Jul 08 23:03:49.460255 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.460125 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f9a58df3-ab7d-4b82-9e17-b7b140727a4b-config-volume\") pod \"dns-default-jb42p\" (UID: \"f9a58df3-ab7d-4b82-9e17-b7b140727a4b\") " pod="openshift-dns/dns-default-jb42p" Jul 08 23:03:49.460499 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.460314 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9a58df3-ab7d-4b82-9e17-b7b140727a4b-metrics-tls\") pod \"dns-default-jb42p\" (UID: \"f9a58df3-ab7d-4b82-9e17-b7b140727a4b\") " pod="openshift-dns/dns-default-jb42p" Jul 08 23:03:49.460499 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.460368 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv8pl\" (UniqueName: \"kubernetes.io/projected/f9a58df3-ab7d-4b82-9e17-b7b140727a4b-kube-api-access-xv8pl\") pod \"dns-default-jb42p\" (UID: \"f9a58df3-ab7d-4b82-9e17-b7b140727a4b\") " pod="openshift-dns/dns-default-jb42p" Jul 08 23:03:49.460499 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.460445 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9a58df3-ab7d-4b82-9e17-b7b140727a4b-tmp-dir\") pod \"dns-default-jb42p\" (UID: \"f9a58df3-ab7d-4b82-9e17-b7b140727a4b\") " pod="openshift-dns/dns-default-jb42p" Jul 08 23:03:49.460499 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.460464 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hsjss" Jul 08 23:03:49.460811 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.460791 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f9a58df3-ab7d-4b82-9e17-b7b140727a4b-config-volume\") pod \"dns-default-jb42p\" (UID: \"f9a58df3-ab7d-4b82-9e17-b7b140727a4b\") " pod="openshift-dns/dns-default-jb42p" Jul 08 23:03:49.462503 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.462483 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9a58df3-ab7d-4b82-9e17-b7b140727a4b-metrics-tls\") pod \"dns-default-jb42p\" (UID: \"f9a58df3-ab7d-4b82-9e17-b7b140727a4b\") " pod="openshift-dns/dns-default-jb42p" Jul 08 23:03:49.467964 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.467940 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv8pl\" (UniqueName: \"kubernetes.io/projected/f9a58df3-ab7d-4b82-9e17-b7b140727a4b-kube-api-access-xv8pl\") pod \"dns-default-jb42p\" (UID: \"f9a58df3-ab7d-4b82-9e17-b7b140727a4b\") " pod="openshift-dns/dns-default-jb42p" Jul 08 23:03:49.468107 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.468093 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x4nvj" Jul 08 23:03:49.589415 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.589090 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jb42p" Jul 08 23:03:49.607250 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.607156 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cc7b69d54-6vkwp"] Jul 08 23:03:49.610786 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:03:49.610732 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fc2cc5c_7632_4171_9813_d13c9b038424.slice/crio-94ca22cb1ee5074ba8cbefba248b8b65eb062a35483ee8d519c16aa8070d2a86 WatchSource:0}: Error finding container 94ca22cb1ee5074ba8cbefba248b8b65eb062a35483ee8d519c16aa8070d2a86: Status 404 returned error can't find the container with id 94ca22cb1ee5074ba8cbefba248b8b65eb062a35483ee8d519c16aa8070d2a86 Jul 08 23:03:49.619397 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.619377 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hsjss"] Jul 08 23:03:49.623132 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:03:49.623108 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8fae214_eb17_4ee8_b4d2_de209a11ad64.slice/crio-6541c6e1dbf2b5d41f235c1d42705eef3b7b9583418b603dd3f7bd008ced31cf WatchSource:0}: Error finding container 6541c6e1dbf2b5d41f235c1d42705eef3b7b9583418b603dd3f7bd008ced31cf: Status 404 returned error can't find the container with id 6541c6e1dbf2b5d41f235c1d42705eef3b7b9583418b603dd3f7bd008ced31cf Jul 08 23:03:49.638876 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.638812 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x4nvj"] Jul 08 23:03:49.653196 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:03:49.653164 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod577bedd6_568d_4e24_ab8e_0ed9ae106385.slice/crio-e16727508b450487cf8dd63c7aeb7c9a04c7814a61952fab3e4bab0beaf7be96 WatchSource:0}: Error finding container e16727508b450487cf8dd63c7aeb7c9a04c7814a61952fab3e4bab0beaf7be96: Status 404 returned error can't find the container with id e16727508b450487cf8dd63c7aeb7c9a04c7814a61952fab3e4bab0beaf7be96 Jul 08 23:03:49.743227 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.743195 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jb42p"] Jul 08 23:03:49.746222 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:03:49.746189 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9a58df3_ab7d_4b82_9e17_b7b140727a4b.slice/crio-7630ada7c56013c1de562560f7b0dd4c7bec67073948c6b232512178f0b326cc WatchSource:0}: Error finding container 7630ada7c56013c1de562560f7b0dd4c7bec67073948c6b232512178f0b326cc: Status 404 returned error can't find the container with id 7630ada7c56013c1de562560f7b0dd4c7bec67073948c6b232512178f0b326cc Jul 08 23:03:49.823528 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.823491 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:03:49.826170 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.826151 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 23:03:49.826485 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:49.826181 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8bt8r\"" Jul 08 23:03:50.137670 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:50.137629 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jb42p" event={"ID":"f9a58df3-ab7d-4b82-9e17-b7b140727a4b","Type":"ContainerStarted","Data":"7630ada7c56013c1de562560f7b0dd4c7bec67073948c6b232512178f0b326cc"} Jul 08 23:03:50.139766 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:50.139734 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" event={"ID":"1fc2cc5c-7632-4171-9813-d13c9b038424","Type":"ContainerStarted","Data":"73feda254222078d5969d763321ef0c21b2b88eefd01cde3b8188eeda7dc2f5b"} Jul 08 23:03:50.139906 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:50.139790 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" event={"ID":"1fc2cc5c-7632-4171-9813-d13c9b038424","Type":"ContainerStarted","Data":"94ca22cb1ee5074ba8cbefba248b8b65eb062a35483ee8d519c16aa8070d2a86"} Jul 08 23:03:50.139906 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:50.139854 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:03:50.141870 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:50.141810 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x4nvj" event={"ID":"577bedd6-568d-4e24-ab8e-0ed9ae106385","Type":"ContainerStarted","Data":"e16727508b450487cf8dd63c7aeb7c9a04c7814a61952fab3e4bab0beaf7be96"} Jul 08 23:03:50.143317 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:50.143182 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hsjss" event={"ID":"b8fae214-eb17-4ee8-b4d2-de209a11ad64","Type":"ContainerStarted","Data":"6541c6e1dbf2b5d41f235c1d42705eef3b7b9583418b603dd3f7bd008ced31cf"} Jul 08 23:03:50.160315 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:50.159973 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" podStartSLOduration=2.15995589 podStartE2EDuration="2.15995589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:03:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:50.158394823 +0000 UTC m=+51.918889891" watchObservedRunningTime="2026-07-08 23:03:50.15995589 +0000 UTC m=+51.920450929" Jul 08 23:03:50.823751 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:50.823719 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:03:50.826523 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:50.824366 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:03:50.836890 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:50.836677 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f6f6s\"" Jul 08 23:03:50.837013 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:50.836948 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 23:03:50.837264 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:50.837149 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 23:03:50.837264 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:50.837149 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 23:03:55.159110 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:55.158975 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x4nvj" event={"ID":"577bedd6-568d-4e24-ab8e-0ed9ae106385","Type":"ContainerStarted","Data":"efbc78b847bfec831a3e89b7413725d685a41bf72b93deef57905073c487543b"} Jul 08 23:03:55.161027 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:55.160999 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hsjss" event={"ID":"b8fae214-eb17-4ee8-b4d2-de209a11ad64","Type":"ContainerStarted","Data":"c8f5096cb09bf7e7e930eac252b2c4ca652117ffad42f65ec3a67140476c8766"} Jul 08 23:03:55.165403 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:55.165366 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jb42p" event={"ID":"f9a58df3-ab7d-4b82-9e17-b7b140727a4b","Type":"ContainerStarted","Data":"512e29ac5e9cb753ba5c852da979288fa3d663a5ee555e1b1c9ac80579ba957f"} Jul 08 23:03:55.165534 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:55.165411 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jb42p" event={"ID":"f9a58df3-ab7d-4b82-9e17-b7b140727a4b","Type":"ContainerStarted","Data":"8e0e409e0e5b24df6b1c4042c9664a5dbcb870f7d7eb2ee523bc2b8516790636"} Jul 08 23:03:55.165534 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:55.165499 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jb42p" Jul 08 23:03:55.175257 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:55.175209 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hsjss" podStartSLOduration=0.835781808 podStartE2EDuration="6.175193599s" totalImagesPullingTime="5.339411791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:03:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:55.174731276 +0000 UTC m=+56.935226331" watchObservedRunningTime="2026-07-08 23:03:55.175193599 +0000 UTC m=+56.935688653" Jul 08 23:03:55.190946 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:55.190899 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jb42p" podStartSLOduration=0.974506093 podStartE2EDuration="6.19088139s" totalImagesPullingTime="5.216375297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:03:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:55.190303734 +0000 UTC m=+56.950798789" watchObservedRunningTime="2026-07-08 23:03:55.19088139 +0000 UTC m=+56.951376422" Jul 08 23:03:56.171873 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:56.171838 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x4nvj" event={"ID":"577bedd6-568d-4e24-ab8e-0ed9ae106385","Type":"ContainerStarted","Data":"6c47c28f32c8d53e3675b80ee22e62892bb48e35b08586327286f14d166459e5"} Jul 08 23:03:58.180652 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:58.180619 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x4nvj" event={"ID":"577bedd6-568d-4e24-ab8e-0ed9ae106385","Type":"ContainerStarted","Data":"0f7976fdf9025af38673d1a491e3bb9520d10fe947c1b0355522ef3c77d0aa80"} Jul 08 23:03:58.196922 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:58.196876 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-x4nvj" podStartSLOduration=0.976598246 podStartE2EDuration="9.196860573s" totalImagesPullingTime="8.220262327s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:03:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:58.196508533 +0000 UTC m=+59.957003587" watchObservedRunningTime="2026-07-08 23:03:58.196860573 +0000 UTC m=+59.957355626" Jul 08 23:03:59.094891 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:03:59.094859 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fv2dh" Jul 08 23:04:03.468922 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.468875 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp"] Jul 08 23:04:03.471384 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.471365 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" Jul 08 23:04:03.474964 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.474938 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 23:04:03.475086 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.474938 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 23:04:03.475086 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.474939 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 08 23:04:03.475086 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.474939 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 23:04:03.476016 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.475997 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 08 23:04:03.476105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.476003 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-xxb4l\"" Jul 08 23:04:03.482963 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.482941 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp"] Jul 08 23:04:03.508674 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.508645 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8"] Jul 08 23:04:03.510636 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.510621 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.513138 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.513116 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 08 23:04:03.513253 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.513197 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 08 23:04:03.513575 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.513560 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 08 23:04:03.513680 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.513664 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-52n5m\"" Jul 08 23:04:03.518154 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.518131 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6dl92"] Jul 08 23:04:03.520680 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.520665 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.523464 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.523445 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 23:04:03.523570 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.523517 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sc5qc\"" Jul 08 23:04:03.523570 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.523542 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 23:04:03.523852 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.523835 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 23:04:03.524790 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.524772 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8"] Jul 08 23:04:03.563429 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.563398 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/13324038-f77f-43d4-9b13-7c8935e64c5b-node-exporter-wtmp\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.563616 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.563463 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eccab368-9199-4510-a77e-6909f8116c1e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-ncgc8\" (UID: \"eccab368-9199-4510-a77e-6909f8116c1e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.563616 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.563505 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/88e9bf6f-ea85-45ee-970a-a7e4632e477d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-fp2bp\" (UID: \"88e9bf6f-ea85-45ee-970a-a7e4632e477d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" Jul 08 23:04:03.563616 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.563547 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/13324038-f77f-43d4-9b13-7c8935e64c5b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.563781 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.563620 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret\") pod \"global-pull-secret-syncer-dwldq\" (UID: \"213525b4-d967-437a-9c90-d40f2190a5a5\") " pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:04:03.563781 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.563664 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/13324038-f77f-43d4-9b13-7c8935e64c5b-node-exporter-accelerators-collector-config\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.563781 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.563706 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eccab368-9199-4510-a77e-6909f8116c1e-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-ncgc8\" (UID: \"eccab368-9199-4510-a77e-6909f8116c1e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.563781 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.563742 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/eccab368-9199-4510-a77e-6909f8116c1e-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-ncgc8\" (UID: \"eccab368-9199-4510-a77e-6909f8116c1e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.563781 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.563773 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13324038-f77f-43d4-9b13-7c8935e64c5b-metrics-client-ca\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.564131 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.563804 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/88e9bf6f-ea85-45ee-970a-a7e4632e477d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fp2bp\" (UID: \"88e9bf6f-ea85-45ee-970a-a7e4632e477d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" Jul 08 23:04:03.564131 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.563838 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/13324038-f77f-43d4-9b13-7c8935e64c5b-root\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.564131 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.563868 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4w6nk\" (UniqueName: \"kubernetes.io/projected/88e9bf6f-ea85-45ee-970a-a7e4632e477d-kube-api-access-4w6nk\") pod \"openshift-state-metrics-54cf96dfcf-fp2bp\" (UID: \"88e9bf6f-ea85-45ee-970a-a7e4632e477d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" Jul 08 23:04:03.564131 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.563903 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eccab368-9199-4510-a77e-6909f8116c1e-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-ncgc8\" (UID: \"eccab368-9199-4510-a77e-6909f8116c1e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.564131 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.563941 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs\") pod \"network-metrics-daemon-4qwwh\" (UID: \"48abd27d-5a99-4c70-937b-b9d06f033843\") " pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:04:03.564131 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.563969 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/13324038-f77f-43d4-9b13-7c8935e64c5b-node-exporter-tls\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.564131 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.564001 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/eccab368-9199-4510-a77e-6909f8116c1e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-ncgc8\" (UID: \"eccab368-9199-4510-a77e-6909f8116c1e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.564131 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.564032 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/13324038-f77f-43d4-9b13-7c8935e64c5b-node-exporter-textfile\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.564131 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.564062 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88e9bf6f-ea85-45ee-970a-a7e4632e477d-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-fp2bp\" (UID: \"88e9bf6f-ea85-45ee-970a-a7e4632e477d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" Jul 08 23:04:03.564131 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.564119 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd7gn\" (UniqueName: \"kubernetes.io/projected/eccab368-9199-4510-a77e-6909f8116c1e-kube-api-access-pd7gn\") pod \"kube-state-metrics-7c9dbb8b7d-ncgc8\" (UID: \"eccab368-9199-4510-a77e-6909f8116c1e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.564553 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.564153 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g4d2\" (UniqueName: \"kubernetes.io/projected/13324038-f77f-43d4-9b13-7c8935e64c5b-kube-api-access-6g4d2\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.564553 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.564182 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13324038-f77f-43d4-9b13-7c8935e64c5b-sys\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.567208 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.567178 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 23:04:03.567208 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.567178 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 23:04:03.577540 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.577512 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/213525b4-d967-437a-9c90-d40f2190a5a5-original-pull-secret\") pod \"global-pull-secret-syncer-dwldq\" (UID: \"213525b4-d967-437a-9c90-d40f2190a5a5\") " pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:04:03.577710 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.577664 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48abd27d-5a99-4c70-937b-b9d06f033843-metrics-certs\") pod \"network-metrics-daemon-4qwwh\" (UID: \"48abd27d-5a99-4c70-937b-b9d06f033843\") " pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:04:03.634982 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.634959 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8bt8r\"" Jul 08 23:04:03.643009 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.642984 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qwwh" Jul 08 23:04:03.665331 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.665297 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd7gn\" (UniqueName: \"kubernetes.io/projected/eccab368-9199-4510-a77e-6909f8116c1e-kube-api-access-pd7gn\") pod \"kube-state-metrics-7c9dbb8b7d-ncgc8\" (UID: \"eccab368-9199-4510-a77e-6909f8116c1e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.665443 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.665345 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g4d2\" (UniqueName: \"kubernetes.io/projected/13324038-f77f-43d4-9b13-7c8935e64c5b-kube-api-access-6g4d2\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.665443 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.665374 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13324038-f77f-43d4-9b13-7c8935e64c5b-sys\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.665443 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.665398 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/13324038-f77f-43d4-9b13-7c8935e64c5b-node-exporter-wtmp\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.665443 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.665429 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eccab368-9199-4510-a77e-6909f8116c1e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-ncgc8\" (UID: \"eccab368-9199-4510-a77e-6909f8116c1e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.665608 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.665456 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/88e9bf6f-ea85-45ee-970a-a7e4632e477d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-fp2bp\" (UID: \"88e9bf6f-ea85-45ee-970a-a7e4632e477d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" Jul 08 23:04:03.665608 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.665494 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13324038-f77f-43d4-9b13-7c8935e64c5b-sys\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.665608 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.665507 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/13324038-f77f-43d4-9b13-7c8935e64c5b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.665710 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.665644 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/13324038-f77f-43d4-9b13-7c8935e64c5b-node-exporter-accelerators-collector-config\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.665710 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.665676 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/13324038-f77f-43d4-9b13-7c8935e64c5b-node-exporter-wtmp\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.665710 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.665702 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eccab368-9199-4510-a77e-6909f8116c1e-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-ncgc8\" (UID: \"eccab368-9199-4510-a77e-6909f8116c1e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.665866 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.665834 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/eccab368-9199-4510-a77e-6909f8116c1e-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-ncgc8\" (UID: \"eccab368-9199-4510-a77e-6909f8116c1e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.665939 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.665882 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13324038-f77f-43d4-9b13-7c8935e64c5b-metrics-client-ca\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.665939 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.665923 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/88e9bf6f-ea85-45ee-970a-a7e4632e477d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fp2bp\" (UID: \"88e9bf6f-ea85-45ee-970a-a7e4632e477d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" Jul 08 23:04:03.666102 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.665963 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/13324038-f77f-43d4-9b13-7c8935e64c5b-root\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.666102 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.665988 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w6nk\" (UniqueName: \"kubernetes.io/projected/88e9bf6f-ea85-45ee-970a-a7e4632e477d-kube-api-access-4w6nk\") pod \"openshift-state-metrics-54cf96dfcf-fp2bp\" (UID: \"88e9bf6f-ea85-45ee-970a-a7e4632e477d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" Jul 08 23:04:03.666102 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.666017 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eccab368-9199-4510-a77e-6909f8116c1e-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-ncgc8\" (UID: \"eccab368-9199-4510-a77e-6909f8116c1e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.666102 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.666047 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/13324038-f77f-43d4-9b13-7c8935e64c5b-node-exporter-tls\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.666102 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.666077 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/eccab368-9199-4510-a77e-6909f8116c1e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-ncgc8\" (UID: \"eccab368-9199-4510-a77e-6909f8116c1e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.666102 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:04:03.666086 2548 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 08 23:04:03.666383 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.666104 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/13324038-f77f-43d4-9b13-7c8935e64c5b-node-exporter-textfile\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.666383 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.666131 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88e9bf6f-ea85-45ee-970a-a7e4632e477d-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-fp2bp\" (UID: \"88e9bf6f-ea85-45ee-970a-a7e4632e477d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" Jul 08 23:04:03.666383 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:04:03.666167 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e9bf6f-ea85-45ee-970a-a7e4632e477d-openshift-state-metrics-tls podName:88e9bf6f-ea85-45ee-970a-a7e4632e477d nodeName:}" failed. No retries permitted until 2026-07-08 23:04:04.166143222 +0000 UTC m=+65.926638265 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/88e9bf6f-ea85-45ee-970a-a7e4632e477d-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-fp2bp" (UID: "88e9bf6f-ea85-45ee-970a-a7e4632e477d") : secret "openshift-state-metrics-tls" not found Jul 08 23:04:03.666383 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.666194 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t67kc\" (UniqueName: \"kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc\") pod \"network-check-target-chcsc\" (UID: \"825c51de-1ee6-48cd-8e9c-66811e605fbf\") " pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:04:03.666383 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.666288 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/eccab368-9199-4510-a77e-6909f8116c1e-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-ncgc8\" (UID: \"eccab368-9199-4510-a77e-6909f8116c1e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.666647 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.666513 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/13324038-f77f-43d4-9b13-7c8935e64c5b-root\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.666708 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.666645 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/13324038-f77f-43d4-9b13-7c8935e64c5b-node-exporter-accelerators-collector-config\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.667063 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.666882 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88e9bf6f-ea85-45ee-970a-a7e4632e477d-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-fp2bp\" (UID: \"88e9bf6f-ea85-45ee-970a-a7e4632e477d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" Jul 08 23:04:03.667063 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:04:03.666977 2548 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 23:04:03.667063 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.666998 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/13324038-f77f-43d4-9b13-7c8935e64c5b-node-exporter-textfile\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.667063 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:04:03.667027 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13324038-f77f-43d4-9b13-7c8935e64c5b-node-exporter-tls podName:13324038-f77f-43d4-9b13-7c8935e64c5b nodeName:}" failed. No retries permitted until 2026-07-08 23:04:04.167009873 +0000 UTC m=+65.927504939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/13324038-f77f-43d4-9b13-7c8935e64c5b-node-exporter-tls") pod "node-exporter-6dl92" (UID: "13324038-f77f-43d4-9b13-7c8935e64c5b") : secret "node-exporter-tls" not found Jul 08 23:04:03.667063 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.667037 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13324038-f77f-43d4-9b13-7c8935e64c5b-metrics-client-ca\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.667446 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.667411 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eccab368-9199-4510-a77e-6909f8116c1e-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-ncgc8\" (UID: \"eccab368-9199-4510-a77e-6909f8116c1e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.667813 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.667766 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/eccab368-9199-4510-a77e-6909f8116c1e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-ncgc8\" (UID: \"eccab368-9199-4510-a77e-6909f8116c1e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.668412 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.668380 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eccab368-9199-4510-a77e-6909f8116c1e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-ncgc8\" (UID: \"eccab368-9199-4510-a77e-6909f8116c1e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.668412 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.668392 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/88e9bf6f-ea85-45ee-970a-a7e4632e477d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-fp2bp\" (UID: \"88e9bf6f-ea85-45ee-970a-a7e4632e477d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" Jul 08 23:04:03.669452 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.669434 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/13324038-f77f-43d4-9b13-7c8935e64c5b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.669555 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.669435 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 23:04:03.669555 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.669500 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eccab368-9199-4510-a77e-6909f8116c1e-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-ncgc8\" (UID: \"eccab368-9199-4510-a77e-6909f8116c1e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.680740 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.680059 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 23:04:03.681273 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.681250 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd7gn\" (UniqueName: \"kubernetes.io/projected/eccab368-9199-4510-a77e-6909f8116c1e-kube-api-access-pd7gn\") pod \"kube-state-metrics-7c9dbb8b7d-ncgc8\" (UID: \"eccab368-9199-4510-a77e-6909f8116c1e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.681367 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.681299 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4w6nk\" (UniqueName: \"kubernetes.io/projected/88e9bf6f-ea85-45ee-970a-a7e4632e477d-kube-api-access-4w6nk\") pod \"openshift-state-metrics-54cf96dfcf-fp2bp\" (UID: \"88e9bf6f-ea85-45ee-970a-a7e4632e477d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" Jul 08 23:04:03.682249 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.682229 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g4d2\" (UniqueName: \"kubernetes.io/projected/13324038-f77f-43d4-9b13-7c8935e64c5b-kube-api-access-6g4d2\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:03.691212 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.691189 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t67kc\" (UniqueName: \"kubernetes.io/projected/825c51de-1ee6-48cd-8e9c-66811e605fbf-kube-api-access-t67kc\") pod \"network-check-target-chcsc\" (UID: \"825c51de-1ee6-48cd-8e9c-66811e605fbf\") " pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:04:03.741982 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.741953 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dwldq" Jul 08 23:04:03.753147 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.753126 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f6f6s\"" Jul 08 23:04:03.761226 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.761201 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:04:03.763656 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.763444 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4qwwh"] Jul 08 23:04:03.768793 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:04:03.768759 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48abd27d_5a99_4c70_937b_b9d06f033843.slice/crio-e6477f4bb87618958d9390d7e87267e74b8772094db491fb5243845cc4ef550c WatchSource:0}: Error finding container e6477f4bb87618958d9390d7e87267e74b8772094db491fb5243845cc4ef550c: Status 404 returned error can't find the container with id e6477f4bb87618958d9390d7e87267e74b8772094db491fb5243845cc4ef550c Jul 08 23:04:03.819882 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.819848 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" Jul 08 23:04:03.887182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.887129 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dwldq"] Jul 08 23:04:03.891957 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:04:03.891931 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod213525b4_d967_437a_9c90_d40f2190a5a5.slice/crio-819501133f1fdcac77bc5e2cbb1735f5d8f393189bf5dee09dba2669a9a7e594 WatchSource:0}: Error finding container 819501133f1fdcac77bc5e2cbb1735f5d8f393189bf5dee09dba2669a9a7e594: Status 404 returned error can't find the container with id 819501133f1fdcac77bc5e2cbb1735f5d8f393189bf5dee09dba2669a9a7e594 Jul 08 23:04:03.903007 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.902980 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-chcsc"] Jul 08 23:04:03.906647 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:04:03.906616 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod825c51de_1ee6_48cd_8e9c_66811e605fbf.slice/crio-498e1521a627b2a52c6a17c9aee2fc52d111608cb8b30f071c3720745de43ad2 WatchSource:0}: Error finding container 498e1521a627b2a52c6a17c9aee2fc52d111608cb8b30f071c3720745de43ad2: Status 404 returned error can't find the container with id 498e1521a627b2a52c6a17c9aee2fc52d111608cb8b30f071c3720745de43ad2 Jul 08 23:04:03.957912 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:03.957875 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8"] Jul 08 23:04:03.960759 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:04:03.960735 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeccab368_9199_4510_a77e_6909f8116c1e.slice/crio-fbf04c46f8c640e9136c2fb3fcbce3ee91d424f8a9a3caef7243f0fbaad7b657 WatchSource:0}: Error finding container fbf04c46f8c640e9136c2fb3fcbce3ee91d424f8a9a3caef7243f0fbaad7b657: Status 404 returned error can't find the container with id fbf04c46f8c640e9136c2fb3fcbce3ee91d424f8a9a3caef7243f0fbaad7b657 Jul 08 23:04:04.170062 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:04.170033 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/13324038-f77f-43d4-9b13-7c8935e64c5b-node-exporter-tls\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:04.170245 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:04.170106 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/88e9bf6f-ea85-45ee-970a-a7e4632e477d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fp2bp\" (UID: \"88e9bf6f-ea85-45ee-970a-a7e4632e477d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" Jul 08 23:04:04.172375 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:04.172344 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/13324038-f77f-43d4-9b13-7c8935e64c5b-node-exporter-tls\") pod \"node-exporter-6dl92\" (UID: \"13324038-f77f-43d4-9b13-7c8935e64c5b\") " pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:04.172475 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:04.172378 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/88e9bf6f-ea85-45ee-970a-a7e4632e477d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fp2bp\" (UID: \"88e9bf6f-ea85-45ee-970a-a7e4632e477d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" Jul 08 23:04:04.198758 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:04.198721 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" event={"ID":"eccab368-9199-4510-a77e-6909f8116c1e","Type":"ContainerStarted","Data":"fbf04c46f8c640e9136c2fb3fcbce3ee91d424f8a9a3caef7243f0fbaad7b657"} Jul 08 23:04:04.199817 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:04.199797 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-chcsc" event={"ID":"825c51de-1ee6-48cd-8e9c-66811e605fbf","Type":"ContainerStarted","Data":"498e1521a627b2a52c6a17c9aee2fc52d111608cb8b30f071c3720745de43ad2"} Jul 08 23:04:04.200989 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:04.200958 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4qwwh" event={"ID":"48abd27d-5a99-4c70-937b-b9d06f033843","Type":"ContainerStarted","Data":"e6477f4bb87618958d9390d7e87267e74b8772094db491fb5243845cc4ef550c"} Jul 08 23:04:04.201939 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:04.201914 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dwldq" event={"ID":"213525b4-d967-437a-9c90-d40f2190a5a5","Type":"ContainerStarted","Data":"819501133f1fdcac77bc5e2cbb1735f5d8f393189bf5dee09dba2669a9a7e594"} Jul 08 23:04:04.380312 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:04.380277 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" Jul 08 23:04:04.429845 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:04.429761 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6dl92" Jul 08 23:04:04.549175 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:04.549137 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp"] Jul 08 23:04:04.555166 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:04:04.555133 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88e9bf6f_ea85_45ee_970a_a7e4632e477d.slice/crio-f9b6879b24ad537864ac5e6702ade62dd294fd2eb75012d6c2cc61e01abda65b WatchSource:0}: Error finding container f9b6879b24ad537864ac5e6702ade62dd294fd2eb75012d6c2cc61e01abda65b: Status 404 returned error can't find the container with id f9b6879b24ad537864ac5e6702ade62dd294fd2eb75012d6c2cc61e01abda65b Jul 08 23:04:05.174149 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:05.173857 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jb42p" Jul 08 23:04:05.211461 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:05.211384 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6dl92" event={"ID":"13324038-f77f-43d4-9b13-7c8935e64c5b","Type":"ContainerStarted","Data":"eab21573965c21947179cd960314178a2b7e170a356a57ed8bd39b711110194e"} Jul 08 23:04:05.215573 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:05.215403 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" event={"ID":"88e9bf6f-ea85-45ee-970a-a7e4632e477d","Type":"ContainerStarted","Data":"a195a3b75410058f21bedc301a11c4fb2810de4bf242c70a082ab600b4160f6f"} Jul 08 23:04:05.215573 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:05.215464 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" event={"ID":"88e9bf6f-ea85-45ee-970a-a7e4632e477d","Type":"ContainerStarted","Data":"e7ea7d0d5d971ac1b21eaa6f4ab2a0437e2a5c949f547835b615cf495015e529"} Jul 08 23:04:05.215573 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:05.215492 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" event={"ID":"88e9bf6f-ea85-45ee-970a-a7e4632e477d","Type":"ContainerStarted","Data":"f9b6879b24ad537864ac5e6702ade62dd294fd2eb75012d6c2cc61e01abda65b"} Jul 08 23:04:06.221173 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:06.221136 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4qwwh" event={"ID":"48abd27d-5a99-4c70-937b-b9d06f033843","Type":"ContainerStarted","Data":"cbb60ed2684a1eaf7154a1aa61db7975eaa5fa931a4d561726f2cf54bd60cef4"} Jul 08 23:04:06.221615 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:06.221197 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4qwwh" event={"ID":"48abd27d-5a99-4c70-937b-b9d06f033843","Type":"ContainerStarted","Data":"2af2d77f28546c5a1136e3cb6cd9ae7406d04eb6398c8624a108cb372bae1d4e"} Jul 08 23:04:06.238046 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:06.237989 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4qwwh" podStartSLOduration=65.92988514 podStartE2EDuration="1m7.2379687s" totalImagesPullingTime="1.30808356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:02:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:06.237239179 +0000 UTC m=+67.997734236" watchObservedRunningTime="2026-07-08 23:04:06.2379687 +0000 UTC m=+67.998463758" Jul 08 23:04:07.227036 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:07.226994 2548 generic.go:356] "Generic (PLEG): container finished" podID="13324038-f77f-43d4-9b13-7c8935e64c5b" containerID="8896d7f29a8b56b5deae5da4ac458ea0adf764a89cc4d877e9c686adb27116af" exitCode=0 Jul 08 23:04:07.227480 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:07.227087 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6dl92" event={"ID":"13324038-f77f-43d4-9b13-7c8935e64c5b","Type":"ContainerDied","Data":"8896d7f29a8b56b5deae5da4ac458ea0adf764a89cc4d877e9c686adb27116af"} Jul 08 23:04:07.228826 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:07.228805 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" event={"ID":"eccab368-9199-4510-a77e-6909f8116c1e","Type":"ContainerStarted","Data":"b32d8a6b710cfab4512e0013d6cf933d5ae7d86e1011be78358cce9e5df07e8f"} Jul 08 23:04:08.192186 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:08.192153 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-bkr4d"] Jul 08 23:04:08.194515 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:08.194446 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bkr4d" Jul 08 23:04:08.197131 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:08.197107 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-scr7h\"" Jul 08 23:04:08.197274 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:08.197171 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 08 23:04:08.203354 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:08.203301 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-bkr4d"] Jul 08 23:04:08.303628 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:08.303573 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvhmb\" (UniqueName: \"kubernetes.io/projected/1cf33f7c-f467-48d7-99d7-2a93d7b55583-kube-api-access-mvhmb\") pod \"monitoring-plugin-d5dcff7cf-bkr4d\" (UID: \"1cf33f7c-f467-48d7-99d7-2a93d7b55583\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bkr4d" Jul 08 23:04:08.304012 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:08.303638 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1cf33f7c-f467-48d7-99d7-2a93d7b55583-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-bkr4d\" (UID: \"1cf33f7c-f467-48d7-99d7-2a93d7b55583\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bkr4d" Jul 08 23:04:08.404717 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:08.404681 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvhmb\" (UniqueName: \"kubernetes.io/projected/1cf33f7c-f467-48d7-99d7-2a93d7b55583-kube-api-access-mvhmb\") pod \"monitoring-plugin-d5dcff7cf-bkr4d\" (UID: \"1cf33f7c-f467-48d7-99d7-2a93d7b55583\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bkr4d" Jul 08 23:04:08.404907 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:08.404734 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1cf33f7c-f467-48d7-99d7-2a93d7b55583-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-bkr4d\" (UID: \"1cf33f7c-f467-48d7-99d7-2a93d7b55583\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bkr4d" Jul 08 23:04:08.407370 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:08.407341 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1cf33f7c-f467-48d7-99d7-2a93d7b55583-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-bkr4d\" (UID: \"1cf33f7c-f467-48d7-99d7-2a93d7b55583\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bkr4d" Jul 08 23:04:08.413213 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:08.413186 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvhmb\" (UniqueName: \"kubernetes.io/projected/1cf33f7c-f467-48d7-99d7-2a93d7b55583-kube-api-access-mvhmb\") pod \"monitoring-plugin-d5dcff7cf-bkr4d\" (UID: \"1cf33f7c-f467-48d7-99d7-2a93d7b55583\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bkr4d" Jul 08 23:04:08.506191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:08.506157 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bkr4d" Jul 08 23:04:09.520400 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:09.520371 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-bkr4d"] Jul 08 23:04:09.525288 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:04:09.525258 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cf33f7c_f467_48d7_99d7_2a93d7b55583.slice/crio-8171390bdaae8aa4001925a17c542a26a7caf8e32b14cad54c4edb02e85e774c WatchSource:0}: Error finding container 8171390bdaae8aa4001925a17c542a26a7caf8e32b14cad54c4edb02e85e774c: Status 404 returned error can't find the container with id 8171390bdaae8aa4001925a17c542a26a7caf8e32b14cad54c4edb02e85e774c Jul 08 23:04:10.241578 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.241536 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" event={"ID":"88e9bf6f-ea85-45ee-970a-a7e4632e477d","Type":"ContainerStarted","Data":"afac355661ecad5d10d90c622d589b9744c18d5b0665b835990d6a11c871d187"} Jul 08 23:04:10.243235 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.243205 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-chcsc" event={"ID":"825c51de-1ee6-48cd-8e9c-66811e605fbf","Type":"ContainerStarted","Data":"a042ebfcbe3cc488c24de7858790261474342371b25e15937120f9689d2ed292"} Jul 08 23:04:10.243398 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.243312 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:04:10.245849 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.245817 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6dl92" event={"ID":"13324038-f77f-43d4-9b13-7c8935e64c5b","Type":"ContainerStarted","Data":"e9dbfdaa37daa11b865c5cbf63ca882e598eb0fc34d6845a5f8a57911438f119"} Jul 08 23:04:10.245849 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.245862 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6dl92" event={"ID":"13324038-f77f-43d4-9b13-7c8935e64c5b","Type":"ContainerStarted","Data":"2d40106036f24667927b80aa3155156e03c66c3d795e5898ee2c830a41d3f299"} Jul 08 23:04:10.247190 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.247167 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dwldq" event={"ID":"213525b4-d967-437a-9c90-d40f2190a5a5","Type":"ContainerStarted","Data":"9f06b1d0eeb96998a29bcc5ae27f88667546c0a4fbd7212d0602a270643386b7"} Jul 08 23:04:10.248120 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.248102 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bkr4d" event={"ID":"1cf33f7c-f467-48d7-99d7-2a93d7b55583","Type":"ContainerStarted","Data":"8171390bdaae8aa4001925a17c542a26a7caf8e32b14cad54c4edb02e85e774c"} Jul 08 23:04:10.249835 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.249815 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" event={"ID":"eccab368-9199-4510-a77e-6909f8116c1e","Type":"ContainerStarted","Data":"56e7b6e8e3460b681ecfb48d27b91ea62a9776f041281ae957119553123d61d2"} Jul 08 23:04:10.249954 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.249854 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" event={"ID":"eccab368-9199-4510-a77e-6909f8116c1e","Type":"ContainerStarted","Data":"03cd64515ed55ff2a5ae9e66ba4633af7c34ec4647355325792b2ece76ac3b83"} Jul 08 23:04:10.261157 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.261117 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp2bp" podStartSLOduration=3.003580328 podStartE2EDuration="7.261103872s" totalImagesPullingTime="4.257523544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:04:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:10.259277376 +0000 UTC m=+72.019772432" watchObservedRunningTime="2026-07-08 23:04:10.261103872 +0000 UTC m=+72.021598923" Jul 08 23:04:10.274947 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.274892 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dwldq" podStartSLOduration=65.800960442 podStartE2EDuration="1m11.274874086s" totalImagesPullingTime="5.473913644s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:02:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:10.273963374 +0000 UTC m=+72.034458429" watchObservedRunningTime="2026-07-08 23:04:10.274874086 +0000 UTC m=+72.035369141" Jul 08 23:04:10.300653 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.300425 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6dl92" podStartSLOduration=5.11032352 podStartE2EDuration="7.300407545s" totalImagesPullingTime="2.080282261s" totalInitContainerRuntime="109.801764ms" isStatefulPod=true podCreationTimestamp="2026-07-08 23:04:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:10.300269132 +0000 UTC m=+72.060764189" watchObservedRunningTime="2026-07-08 23:04:10.300407545 +0000 UTC m=+72.060902600" Jul 08 23:04:10.318616 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.318548 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-chcsc" podStartSLOduration=65.869956739 podStartE2EDuration="1m11.318530486s" totalImagesPullingTime="5.448573747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:02:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:10.318045339 +0000 UTC m=+72.078540395" watchObservedRunningTime="2026-07-08 23:04:10.318530486 +0000 UTC m=+72.079025542" Jul 08 23:04:10.334807 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.334781 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-4686n"] Jul 08 23:04:10.337891 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.337870 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-4686n" Jul 08 23:04:10.343964 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.341837 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 23:04:10.343964 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.342103 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-nwphw\"" Jul 08 23:04:10.343964 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.342128 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ncgc8" podStartSLOduration=4.779744232 podStartE2EDuration="7.342110826s" totalImagesPullingTime="2.562366594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:04:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:10.338888474 +0000 UTC m=+72.099383527" watchObservedRunningTime="2026-07-08 23:04:10.342110826 +0000 UTC m=+72.102605881" Jul 08 23:04:10.343964 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.342339 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 23:04:10.350218 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.350189 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-4686n"] Jul 08 23:04:10.422257 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.422228 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmkd6\" (UniqueName: \"kubernetes.io/projected/091053ae-91a5-41d2-93a5-ecedb8b381f8-kube-api-access-rmkd6\") pod \"downloads-685bfb5f64-4686n\" (UID: \"091053ae-91a5-41d2-93a5-ecedb8b381f8\") " pod="openshift-console/downloads-685bfb5f64-4686n" Jul 08 23:04:10.523328 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.523242 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmkd6\" (UniqueName: \"kubernetes.io/projected/091053ae-91a5-41d2-93a5-ecedb8b381f8-kube-api-access-rmkd6\") pod \"downloads-685bfb5f64-4686n\" (UID: \"091053ae-91a5-41d2-93a5-ecedb8b381f8\") " pod="openshift-console/downloads-685bfb5f64-4686n" Jul 08 23:04:10.531548 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.531520 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmkd6\" (UniqueName: \"kubernetes.io/projected/091053ae-91a5-41d2-93a5-ecedb8b381f8-kube-api-access-rmkd6\") pod \"downloads-685bfb5f64-4686n\" (UID: \"091053ae-91a5-41d2-93a5-ecedb8b381f8\") " pod="openshift-console/downloads-685bfb5f64-4686n" Jul 08 23:04:10.652803 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.652763 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-4686n" Jul 08 23:04:10.993517 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:10.993487 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-4686n"] Jul 08 23:04:10.996578 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:04:10.996546 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091053ae_91a5_41d2_93a5_ecedb8b381f8.slice/crio-ebf0cdbaa87cadd631afb37200ff3347e3e22019c1c291061c796b7d58cc485b WatchSource:0}: Error finding container ebf0cdbaa87cadd631afb37200ff3347e3e22019c1c291061c796b7d58cc485b: Status 404 returned error can't find the container with id ebf0cdbaa87cadd631afb37200ff3347e3e22019c1c291061c796b7d58cc485b Jul 08 23:04:11.152105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:11.152022 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7cc7b69d54-6vkwp" Jul 08 23:04:11.255029 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:11.254993 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bkr4d" event={"ID":"1cf33f7c-f467-48d7-99d7-2a93d7b55583","Type":"ContainerStarted","Data":"83c193059e45f4d70ecbc2bdfbb37255693e3bfa223b9e0733b0ba1a76d5e459"} Jul 08 23:04:11.255220 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:11.255200 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bkr4d" Jul 08 23:04:11.256635 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:11.256584 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-4686n" event={"ID":"091053ae-91a5-41d2-93a5-ecedb8b381f8","Type":"ContainerStarted","Data":"ebf0cdbaa87cadd631afb37200ff3347e3e22019c1c291061c796b7d58cc485b"} Jul 08 23:04:11.260467 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:11.260445 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bkr4d" Jul 08 23:04:11.272286 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:11.272217 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bkr4d" podStartSLOduration=1.8862583160000002 podStartE2EDuration="3.272201914s" totalImagesPullingTime="1.385943598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:04:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:11.271318796 +0000 UTC m=+73.031813853" watchObservedRunningTime="2026-07-08 23:04:11.272201914 +0000 UTC m=+73.032697003" Jul 08 23:04:21.047641 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.047505 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f6d45ff-vjssj"] Jul 08 23:04:21.051817 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.051796 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.054152 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.054123 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 23:04:21.054273 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.054256 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 23:04:21.055268 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.055249 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 23:04:21.055368 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.055322 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dp4fw\"" Jul 08 23:04:21.055368 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.055328 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 23:04:21.055368 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.055363 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 23:04:21.059628 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.059609 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f6d45ff-vjssj"] Jul 08 23:04:21.104608 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.104570 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-console-serving-cert\") pod \"console-f6d45ff-vjssj\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.104778 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.104647 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-oauth-serving-cert\") pod \"console-f6d45ff-vjssj\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.104778 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.104697 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-service-ca\") pod \"console-f6d45ff-vjssj\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.104778 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.104720 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-console-oauth-config\") pod \"console-f6d45ff-vjssj\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.104778 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.104758 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-console-config\") pod \"console-f6d45ff-vjssj\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.104904 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.104805 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2s58\" (UniqueName: \"kubernetes.io/projected/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-kube-api-access-p2s58\") pod \"console-f6d45ff-vjssj\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.205577 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.205542 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-console-serving-cert\") pod \"console-f6d45ff-vjssj\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.205774 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.205589 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-oauth-serving-cert\") pod \"console-f6d45ff-vjssj\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.205774 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.205646 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-service-ca\") pod \"console-f6d45ff-vjssj\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.205774 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.205681 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-console-oauth-config\") pod \"console-f6d45ff-vjssj\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.205774 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.205702 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-console-config\") pod \"console-f6d45ff-vjssj\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.205774 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.205746 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2s58\" (UniqueName: \"kubernetes.io/projected/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-kube-api-access-p2s58\") pod \"console-f6d45ff-vjssj\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.206707 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.206656 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-oauth-serving-cert\") pod \"console-f6d45ff-vjssj\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.206707 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.206686 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-service-ca\") pod \"console-f6d45ff-vjssj\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.206857 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.206714 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-console-config\") pod \"console-f6d45ff-vjssj\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.208548 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.208511 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-console-oauth-config\") pod \"console-f6d45ff-vjssj\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.208548 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.208517 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-console-serving-cert\") pod \"console-f6d45ff-vjssj\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.213927 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.213900 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2s58\" (UniqueName: \"kubernetes.io/projected/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-kube-api-access-p2s58\") pod \"console-f6d45ff-vjssj\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.362470 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.362381 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:21.502230 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:21.502197 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f6d45ff-vjssj"] Jul 08 23:04:21.505567 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:04:21.505533 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2a79ca7_d4da_40d9_b7db_1cbeb1b77ec9.slice/crio-4d80805b8764cae9d1607d8aba1794ba0507cd88423468a46da9966e4e327539 WatchSource:0}: Error finding container 4d80805b8764cae9d1607d8aba1794ba0507cd88423468a46da9966e4e327539: Status 404 returned error can't find the container with id 4d80805b8764cae9d1607d8aba1794ba0507cd88423468a46da9966e4e327539 Jul 08 23:04:22.301745 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:22.301710 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f6d45ff-vjssj" event={"ID":"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9","Type":"ContainerStarted","Data":"4d80805b8764cae9d1607d8aba1794ba0507cd88423468a46da9966e4e327539"} Jul 08 23:04:23.631550 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.631518 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59cb7bf8bc-fzchg"] Jul 08 23:04:23.689651 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.689590 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59cb7bf8bc-fzchg"] Jul 08 23:04:23.689831 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.689698 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.698771 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.698741 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 23:04:23.828014 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.827978 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-trusted-ca-bundle\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.828196 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.828022 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-service-ca\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.828196 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.828052 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-console-oauth-config\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.828196 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.828137 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-oauth-serving-cert\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.828196 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.828193 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-console-config\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.828375 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.828237 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-console-serving-cert\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.828375 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.828256 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx85b\" (UniqueName: \"kubernetes.io/projected/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-kube-api-access-lx85b\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.929378 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.929288 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-console-serving-cert\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.929378 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.929330 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx85b\" (UniqueName: \"kubernetes.io/projected/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-kube-api-access-lx85b\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.929378 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.929361 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-trusted-ca-bundle\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.929686 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.929393 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-service-ca\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.929686 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.929429 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-console-oauth-config\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.929686 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.929616 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-oauth-serving-cert\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.929686 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.929672 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-console-config\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.930426 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.930400 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-service-ca\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.930541 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.930508 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-oauth-serving-cert\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.931014 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.930986 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-console-config\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.931241 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.931218 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-trusted-ca-bundle\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.932032 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.932012 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-console-oauth-config\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.932271 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.932250 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-console-serving-cert\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:23.941681 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:23.941659 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx85b\" (UniqueName: \"kubernetes.io/projected/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-kube-api-access-lx85b\") pod \"console-59cb7bf8bc-fzchg\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:24.001725 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:24.001686 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:29.568649 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:29.568616 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-ncgc8_eccab368-9199-4510-a77e-6909f8116c1e/kube-state-metrics/0.log" Jul 08 23:04:29.767848 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:29.767813 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-ncgc8_eccab368-9199-4510-a77e-6909f8116c1e/kube-rbac-proxy-main/0.log" Jul 08 23:04:29.967021 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:29.966930 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-ncgc8_eccab368-9199-4510-a77e-6909f8116c1e/kube-rbac-proxy-self/0.log" Jul 08 23:04:30.366875 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:30.366844 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-bkr4d_1cf33f7c-f467-48d7-99d7-2a93d7b55583/monitoring-plugin/0.log" Jul 08 23:04:30.567903 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:30.567870 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6dl92_13324038-f77f-43d4-9b13-7c8935e64c5b/init-textfile/0.log" Jul 08 23:04:30.768876 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:30.768849 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6dl92_13324038-f77f-43d4-9b13-7c8935e64c5b/node-exporter/0.log" Jul 08 23:04:30.967571 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:30.967534 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6dl92_13324038-f77f-43d4-9b13-7c8935e64c5b/kube-rbac-proxy/0.log" Jul 08 23:04:32.207263 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:32.207232 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59cb7bf8bc-fzchg"] Jul 08 23:04:32.210695 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:04:32.210663 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a886b8_ac40_4557_a4bc_c5f3b1b22fd8.slice/crio-dc931561cd7fe2ccfb9f08d0c48f71b353a5ecee3ab0eca8d8ba42e3ff789c7f WatchSource:0}: Error finding container dc931561cd7fe2ccfb9f08d0c48f71b353a5ecee3ab0eca8d8ba42e3ff789c7f: Status 404 returned error can't find the container with id dc931561cd7fe2ccfb9f08d0c48f71b353a5ecee3ab0eca8d8ba42e3ff789c7f Jul 08 23:04:32.344762 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:32.344673 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-4686n" event={"ID":"091053ae-91a5-41d2-93a5-ecedb8b381f8","Type":"ContainerStarted","Data":"1c6bb068d7fddd128c0754f80547f89fca68f72daf3d8be2a70895acacf40882"} Jul 08 23:04:32.344929 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:32.344896 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-4686n" Jul 08 23:04:32.346311 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:32.346278 2548 patch_prober.go:28] interesting pod/downloads-685bfb5f64-4686n container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.13:8080/\": dial tcp 10.133.0.13:8080: connect: connection refused" start-of-body= Jul 08 23:04:32.346452 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:32.346333 2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-685bfb5f64-4686n" podUID="091053ae-91a5-41d2-93a5-ecedb8b381f8" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.13:8080/\": dial tcp 10.133.0.13:8080: connect: connection refused" Jul 08 23:04:32.346836 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:32.346803 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f6d45ff-vjssj" event={"ID":"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9","Type":"ContainerStarted","Data":"6365dfb4738940d71d40c20395d7ea723d169d31e9484d19952ce293b7d44c55"} Jul 08 23:04:32.348766 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:32.348738 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59cb7bf8bc-fzchg" event={"ID":"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8","Type":"ContainerStarted","Data":"eb6450cada51c4a4df645624857de9493ebfdebce1c6415bcf1e3e466bc97643"} Jul 08 23:04:32.348903 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:32.348785 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59cb7bf8bc-fzchg" event={"ID":"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8","Type":"ContainerStarted","Data":"dc931561cd7fe2ccfb9f08d0c48f71b353a5ecee3ab0eca8d8ba42e3ff789c7f"} Jul 08 23:04:32.365748 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:32.365684 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-4686n" podStartSLOduration=1.265055673 podStartE2EDuration="22.365664484s" totalImagesPullingTime="21.100608811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:32.363778552 +0000 UTC m=+94.124273602" watchObservedRunningTime="2026-07-08 23:04:32.365664484 +0000 UTC m=+94.126159540" Jul 08 23:04:32.368871 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:32.368844 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fp2bp_88e9bf6f-ea85-45ee-970a-a7e4632e477d/kube-rbac-proxy-main/0.log" Jul 08 23:04:32.382023 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:32.381973 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59cb7bf8bc-fzchg" podStartSLOduration=9.3819597 podStartE2EDuration="9.3819597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:04:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:32.381209396 +0000 UTC m=+94.141704454" watchObservedRunningTime="2026-07-08 23:04:32.3819597 +0000 UTC m=+94.142454733" Jul 08 23:04:32.397161 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:32.397113 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f6d45ff-vjssj" podStartSLOduration=0.822721068 podStartE2EDuration="11.397099326s" totalImagesPullingTime="10.574378258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:04:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:32.396232078 +0000 UTC m=+94.156727135" watchObservedRunningTime="2026-07-08 23:04:32.397099326 +0000 UTC m=+94.157594379" Jul 08 23:04:32.567706 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:32.567673 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fp2bp_88e9bf6f-ea85-45ee-970a-a7e4632e477d/kube-rbac-proxy-self/0.log" Jul 08 23:04:32.767936 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:32.767897 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fp2bp_88e9bf6f-ea85-45ee-970a-a7e4632e477d/openshift-state-metrics/0.log" Jul 08 23:04:33.363199 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:33.363172 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-4686n" Jul 08 23:04:34.002400 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:34.002358 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:34.002579 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:34.002437 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:34.008512 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:34.008479 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:34.362097 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:34.362015 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:04:34.409768 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:34.409733 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f6d45ff-vjssj"] Jul 08 23:04:36.768141 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:36.768097 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-59cb7bf8bc-fzchg_80a886b8-ac40-4557-a4bc-c5f3b1b22fd8/console/0.log" Jul 08 23:04:36.968113 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:36.968025 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-f6d45ff-vjssj_c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9/console/0.log" Jul 08 23:04:37.170275 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:37.170165 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-4686n_091053ae-91a5-41d2-93a5-ecedb8b381f8/download-server/0.log" Jul 08 23:04:37.567383 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:37.567346 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hsjss_b8fae214-eb17-4ee8-b4d2-de209a11ad64/serve-healthcheck-canary/0.log" Jul 08 23:04:41.261330 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:41.261281 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-chcsc" Jul 08 23:04:41.362640 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:41.362581 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:59.435947 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.435881 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-f6d45ff-vjssj" podUID="c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9" containerName="console" containerID="cri-o://6365dfb4738940d71d40c20395d7ea723d169d31e9484d19952ce293b7d44c55" gracePeriod=15 Jul 08 23:04:59.708084 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.708014 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:04:59.804921 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.804879 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-console-oauth-config\") pod \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " Jul 08 23:04:59.805106 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.804941 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-service-ca\") pod \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " Jul 08 23:04:59.805106 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.804969 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2s58\" (UniqueName: \"kubernetes.io/projected/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-kube-api-access-p2s58\") pod \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " Jul 08 23:04:59.805106 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.804996 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-oauth-serving-cert\") pod \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " Jul 08 23:04:59.805106 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.805012 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-console-config\") pod \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " Jul 08 23:04:59.805106 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.805041 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-console-serving-cert\") pod \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\" (UID: \"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9\") " Jul 08 23:04:59.805451 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.805415 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-console-config" (OuterVolumeSpecName: "console-config") pod "c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9" (UID: "c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:04:59.805538 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.805431 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9" (UID: "c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:04:59.805538 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.805428 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-service-ca" (OuterVolumeSpecName: "service-ca") pod "c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9" (UID: "c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:04:59.807179 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.807157 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9" (UID: "c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 23:04:59.807272 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.807191 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-kube-api-access-p2s58" (OuterVolumeSpecName: "kube-api-access-p2s58") pod "c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9" (UID: "c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9"). InnerVolumeSpecName "kube-api-access-p2s58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:04:59.807272 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.807260 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9" (UID: "c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 23:04:59.905833 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.905795 2548 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-service-ca\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:04:59.905833 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.905828 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2s58\" (UniqueName: \"kubernetes.io/projected/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-kube-api-access-p2s58\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:04:59.905833 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.905839 2548 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-oauth-serving-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:04:59.906045 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.905848 2548 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-console-config\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:04:59.906045 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.905857 2548 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-console-serving-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:04:59.906045 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:04:59.905865 2548 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9-console-oauth-config\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:05:00.463464 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:00.463428 2548 generic.go:356] "Generic (PLEG): container finished" podID="c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9" containerID="6365dfb4738940d71d40c20395d7ea723d169d31e9484d19952ce293b7d44c55" exitCode=0 Jul 08 23:05:00.463963 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:00.463476 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f6d45ff-vjssj" Jul 08 23:05:00.463963 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:00.463494 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f6d45ff-vjssj" event={"ID":"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9","Type":"ContainerDied","Data":"6365dfb4738940d71d40c20395d7ea723d169d31e9484d19952ce293b7d44c55"} Jul 08 23:05:00.463963 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:00.463623 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f6d45ff-vjssj" event={"ID":"c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9","Type":"ContainerDied","Data":"4d80805b8764cae9d1607d8aba1794ba0507cd88423468a46da9966e4e327539"} Jul 08 23:05:00.463963 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:00.463652 2548 scope.go:117] "RemoveContainer" containerID="6365dfb4738940d71d40c20395d7ea723d169d31e9484d19952ce293b7d44c55" Jul 08 23:05:00.471968 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:00.471949 2548 scope.go:117] "RemoveContainer" containerID="6365dfb4738940d71d40c20395d7ea723d169d31e9484d19952ce293b7d44c55" Jul 08 23:05:00.472242 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:05:00.472225 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6365dfb4738940d71d40c20395d7ea723d169d31e9484d19952ce293b7d44c55\": container with ID starting with 6365dfb4738940d71d40c20395d7ea723d169d31e9484d19952ce293b7d44c55 not found: ID does not exist" containerID="6365dfb4738940d71d40c20395d7ea723d169d31e9484d19952ce293b7d44c55" Jul 08 23:05:00.472323 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:00.472249 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6365dfb4738940d71d40c20395d7ea723d169d31e9484d19952ce293b7d44c55"} err="failed to get container status \"6365dfb4738940d71d40c20395d7ea723d169d31e9484d19952ce293b7d44c55\": rpc error: code = NotFound desc = could not find container \"6365dfb4738940d71d40c20395d7ea723d169d31e9484d19952ce293b7d44c55\": container with ID starting with 6365dfb4738940d71d40c20395d7ea723d169d31e9484d19952ce293b7d44c55 not found: ID does not exist" Jul 08 23:05:00.483442 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:00.483422 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f6d45ff-vjssj"] Jul 08 23:05:00.487261 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:00.487241 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f6d45ff-vjssj"] Jul 08 23:05:00.831262 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:00.831229 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9" path="/var/lib/kubelet/pods/c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9/volumes" Jul 08 23:05:27.725796 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.725756 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7"] Jul 08 23:05:27.726358 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.726131 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9" containerName="console" Jul 08 23:05:27.726358 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.726147 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9" containerName="console" Jul 08 23:05:27.726358 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.726217 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2a79ca7-d4da-40d9-b7db-1cbeb1b77ec9" containerName="console" Jul 08 23:05:27.728182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.728160 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.730974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.730953 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 23:05:27.731077 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.730988 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 23:05:27.731408 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.731384 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 23:05:27.731524 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.731452 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 23:05:27.731524 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.731486 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 23:05:27.731524 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.731492 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-mgpml\"" Jul 08 23:05:27.737385 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.737366 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 23:05:27.745579 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.745556 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7"] Jul 08 23:05:27.801504 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.801473 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-metrics-client-ca\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.801702 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.801514 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-secret-telemeter-client\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.801702 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.801537 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-telemeter-client-tls\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.801702 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.801630 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.801851 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.801717 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-serving-certs-ca-bundle\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.801851 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.801750 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.801851 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.801781 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-federate-client-tls\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.801851 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.801808 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkcxf\" (UniqueName: \"kubernetes.io/projected/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-kube-api-access-bkcxf\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.902404 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.902367 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.902558 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.902410 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-serving-certs-ca-bundle\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.902558 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.902430 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.902558 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.902446 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-federate-client-tls\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.902558 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.902479 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkcxf\" (UniqueName: \"kubernetes.io/projected/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-kube-api-access-bkcxf\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.902558 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.902503 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-metrics-client-ca\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.902558 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.902556 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-secret-telemeter-client\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.902880 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.902591 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-telemeter-client-tls\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.903309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.903278 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-metrics-client-ca\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.903430 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.903278 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-serving-certs-ca-bundle\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.903510 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.903486 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.905076 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.905052 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-federate-client-tls\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.905210 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.905186 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-secret-telemeter-client\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.905275 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.905258 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.905324 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.905307 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-telemeter-client-tls\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:27.909757 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:27.909736 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkcxf\" (UniqueName: \"kubernetes.io/projected/0d9d05b2-43f2-4dc1-8464-1de07362c7e4-kube-api-access-bkcxf\") pod \"telemeter-client-5fd4fbfbc7-7vvn7\" (UID: \"0d9d05b2-43f2-4dc1-8464-1de07362c7e4\") " pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:28.037639 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:28.037545 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" Jul 08 23:05:28.170959 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:28.170862 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7"] Jul 08 23:05:28.174779 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:05:28.174751 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d9d05b2_43f2_4dc1_8464_1de07362c7e4.slice/crio-9c93453bab0ac3d5f6e4e822a445e189f4167f7195917280fefbaaa7d0d27c39 WatchSource:0}: Error finding container 9c93453bab0ac3d5f6e4e822a445e189f4167f7195917280fefbaaa7d0d27c39: Status 404 returned error can't find the container with id 9c93453bab0ac3d5f6e4e822a445e189f4167f7195917280fefbaaa7d0d27c39 Jul 08 23:05:28.568853 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:28.568818 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" event={"ID":"0d9d05b2-43f2-4dc1-8464-1de07362c7e4","Type":"ContainerStarted","Data":"9c93453bab0ac3d5f6e4e822a445e189f4167f7195917280fefbaaa7d0d27c39"} Jul 08 23:05:30.581910 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:30.581870 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" event={"ID":"0d9d05b2-43f2-4dc1-8464-1de07362c7e4","Type":"ContainerStarted","Data":"0655acd05a8285e672277037552cbdc3d2626d1b060c827eb196509df3af91e0"} Jul 08 23:05:31.587587 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:31.587503 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" event={"ID":"0d9d05b2-43f2-4dc1-8464-1de07362c7e4","Type":"ContainerStarted","Data":"62eaa1f2a23b8765bd7e01606557cde23a05203c3f10323d656b2c03736d93ef"} Jul 08 23:05:31.587587 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:31.587556 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" event={"ID":"0d9d05b2-43f2-4dc1-8464-1de07362c7e4","Type":"ContainerStarted","Data":"83a06fc6566761966432cc48cd8b75b5835e9e308350fa35f1ddec770d2fb096"} Jul 08 23:05:31.609211 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:31.609161 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5fd4fbfbc7-7vvn7" podStartSLOduration=1.6107970520000001 podStartE2EDuration="4.60914552s" totalImagesPullingTime="2.998348468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:05:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:05:31.607288856 +0000 UTC m=+153.367783910" watchObservedRunningTime="2026-07-08 23:05:31.60914552 +0000 UTC m=+153.369640573" Jul 08 23:05:32.482622 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.482574 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d95948d8c-6rn4z"] Jul 08 23:05:32.484660 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.484640 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.499829 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.499786 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d95948d8c-6rn4z"] Jul 08 23:05:32.544562 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.544533 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2wxc\" (UniqueName: \"kubernetes.io/projected/a911fcc2-f1d6-4805-a089-759127502dbd-kube-api-access-k2wxc\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.544562 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.544572 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a911fcc2-f1d6-4805-a089-759127502dbd-console-oauth-config\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.544804 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.544591 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-service-ca\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.544804 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.544697 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-console-config\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.544804 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.544737 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-trusted-ca-bundle\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.544804 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.544765 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a911fcc2-f1d6-4805-a089-759127502dbd-console-serving-cert\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.544968 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.544829 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-oauth-serving-cert\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.645230 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.645199 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-trusted-ca-bundle\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.645723 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.645265 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a911fcc2-f1d6-4805-a089-759127502dbd-console-serving-cert\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.645723 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.645307 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-oauth-serving-cert\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.645723 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.645354 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2wxc\" (UniqueName: \"kubernetes.io/projected/a911fcc2-f1d6-4805-a089-759127502dbd-kube-api-access-k2wxc\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.645723 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.645414 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a911fcc2-f1d6-4805-a089-759127502dbd-console-oauth-config\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.645723 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.645439 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-service-ca\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.645979 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.645850 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-console-config\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.646159 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.646114 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-oauth-serving-cert\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.646159 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.646151 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-service-ca\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.646454 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.646434 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-console-config\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.646584 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.646567 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-trusted-ca-bundle\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.647834 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.647814 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a911fcc2-f1d6-4805-a089-759127502dbd-console-serving-cert\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.647917 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.647892 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a911fcc2-f1d6-4805-a089-759127502dbd-console-oauth-config\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.653326 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.653307 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2wxc\" (UniqueName: \"kubernetes.io/projected/a911fcc2-f1d6-4805-a089-759127502dbd-kube-api-access-k2wxc\") pod \"console-5d95948d8c-6rn4z\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.793576 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.793497 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:32.917566 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:32.917535 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d95948d8c-6rn4z"] Jul 08 23:05:32.920515 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:05:32.920468 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda911fcc2_f1d6_4805_a089_759127502dbd.slice/crio-ce9245bbbca1e59113f9645c7d71d93433ab5883a61d1de9879a2c30088286e6 WatchSource:0}: Error finding container ce9245bbbca1e59113f9645c7d71d93433ab5883a61d1de9879a2c30088286e6: Status 404 returned error can't find the container with id ce9245bbbca1e59113f9645c7d71d93433ab5883a61d1de9879a2c30088286e6 Jul 08 23:05:33.596035 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:33.596000 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d95948d8c-6rn4z" event={"ID":"a911fcc2-f1d6-4805-a089-759127502dbd","Type":"ContainerStarted","Data":"4c94ee6f3913f69e58439789b901f47dfc4f38bf2292a113e3a0dd177d2015bb"} Jul 08 23:05:33.596208 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:33.596080 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d95948d8c-6rn4z" event={"ID":"a911fcc2-f1d6-4805-a089-759127502dbd","Type":"ContainerStarted","Data":"ce9245bbbca1e59113f9645c7d71d93433ab5883a61d1de9879a2c30088286e6"} Jul 08 23:05:33.612497 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:33.612150 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d95948d8c-6rn4z" podStartSLOduration=1.612132966 podStartE2EDuration="1.612132966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:05:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:05:33.611858383 +0000 UTC m=+155.372353440" watchObservedRunningTime="2026-07-08 23:05:33.612132966 +0000 UTC m=+155.372628021" Jul 08 23:05:42.793790 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:42.793754 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:42.794175 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:42.793837 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:42.798622 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:42.798583 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:43.637509 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:43.637480 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:05:43.673803 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:05:43.673771 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59cb7bf8bc-fzchg"] Jul 08 23:06:08.698138 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:08.698076 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-59cb7bf8bc-fzchg" podUID="80a886b8-ac40-4557-a4bc-c5f3b1b22fd8" containerName="console" containerID="cri-o://eb6450cada51c4a4df645624857de9493ebfdebce1c6415bcf1e3e466bc97643" gracePeriod=15 Jul 08 23:06:08.939533 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:08.939502 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:06:09.040261 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.040226 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-service-ca\") pod \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " Jul 08 23:06:09.040451 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.040280 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-oauth-serving-cert\") pod \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " Jul 08 23:06:09.040451 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.040317 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx85b\" (UniqueName: \"kubernetes.io/projected/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-kube-api-access-lx85b\") pod \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " Jul 08 23:06:09.040451 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.040342 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-console-config\") pod \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " Jul 08 23:06:09.040451 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.040361 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-console-oauth-config\") pod \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " Jul 08 23:06:09.040451 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.040396 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-trusted-ca-bundle\") pod \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " Jul 08 23:06:09.040451 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.040434 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-console-serving-cert\") pod \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\" (UID: \"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8\") " Jul 08 23:06:09.040810 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.040726 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-service-ca" (OuterVolumeSpecName: "service-ca") pod "80a886b8-ac40-4557-a4bc-c5f3b1b22fd8" (UID: "80a886b8-ac40-4557-a4bc-c5f3b1b22fd8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:06:09.040810 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.040768 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "80a886b8-ac40-4557-a4bc-c5f3b1b22fd8" (UID: "80a886b8-ac40-4557-a4bc-c5f3b1b22fd8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:06:09.040810 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.040793 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-console-config" (OuterVolumeSpecName: "console-config") pod "80a886b8-ac40-4557-a4bc-c5f3b1b22fd8" (UID: "80a886b8-ac40-4557-a4bc-c5f3b1b22fd8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:06:09.041070 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.041045 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "80a886b8-ac40-4557-a4bc-c5f3b1b22fd8" (UID: "80a886b8-ac40-4557-a4bc-c5f3b1b22fd8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:06:09.042589 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.042565 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "80a886b8-ac40-4557-a4bc-c5f3b1b22fd8" (UID: "80a886b8-ac40-4557-a4bc-c5f3b1b22fd8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 23:06:09.042956 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.042929 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "80a886b8-ac40-4557-a4bc-c5f3b1b22fd8" (UID: "80a886b8-ac40-4557-a4bc-c5f3b1b22fd8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 23:06:09.043025 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.042951 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-kube-api-access-lx85b" (OuterVolumeSpecName: "kube-api-access-lx85b") pod "80a886b8-ac40-4557-a4bc-c5f3b1b22fd8" (UID: "80a886b8-ac40-4557-a4bc-c5f3b1b22fd8"). InnerVolumeSpecName "kube-api-access-lx85b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:06:09.141478 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.141443 2548 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-console-serving-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:06:09.141478 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.141473 2548 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-service-ca\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:06:09.141702 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.141504 2548 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-oauth-serving-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:06:09.141702 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.141513 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx85b\" (UniqueName: \"kubernetes.io/projected/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-kube-api-access-lx85b\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:06:09.141702 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.141523 2548 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-console-config\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:06:09.141702 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.141531 2548 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-console-oauth-config\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:06:09.141702 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.141539 2548 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8-trusted-ca-bundle\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:06:09.725306 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.725274 2548 generic.go:356] "Generic (PLEG): container finished" podID="80a886b8-ac40-4557-a4bc-c5f3b1b22fd8" containerID="eb6450cada51c4a4df645624857de9493ebfdebce1c6415bcf1e3e466bc97643" exitCode=0 Jul 08 23:06:09.725819 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.725313 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59cb7bf8bc-fzchg" event={"ID":"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8","Type":"ContainerDied","Data":"eb6450cada51c4a4df645624857de9493ebfdebce1c6415bcf1e3e466bc97643"} Jul 08 23:06:09.725819 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.725317 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59cb7bf8bc-fzchg" Jul 08 23:06:09.725819 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.725354 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59cb7bf8bc-fzchg" event={"ID":"80a886b8-ac40-4557-a4bc-c5f3b1b22fd8","Type":"ContainerDied","Data":"dc931561cd7fe2ccfb9f08d0c48f71b353a5ecee3ab0eca8d8ba42e3ff789c7f"} Jul 08 23:06:09.725819 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.725374 2548 scope.go:117] "RemoveContainer" containerID="eb6450cada51c4a4df645624857de9493ebfdebce1c6415bcf1e3e466bc97643" Jul 08 23:06:09.738043 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.738026 2548 scope.go:117] "RemoveContainer" containerID="eb6450cada51c4a4df645624857de9493ebfdebce1c6415bcf1e3e466bc97643" Jul 08 23:06:09.738309 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:06:09.738291 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb6450cada51c4a4df645624857de9493ebfdebce1c6415bcf1e3e466bc97643\": container with ID starting with eb6450cada51c4a4df645624857de9493ebfdebce1c6415bcf1e3e466bc97643 not found: ID does not exist" containerID="eb6450cada51c4a4df645624857de9493ebfdebce1c6415bcf1e3e466bc97643" Jul 08 23:06:09.738356 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.738317 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb6450cada51c4a4df645624857de9493ebfdebce1c6415bcf1e3e466bc97643"} err="failed to get container status \"eb6450cada51c4a4df645624857de9493ebfdebce1c6415bcf1e3e466bc97643\": rpc error: code = NotFound desc = could not find container \"eb6450cada51c4a4df645624857de9493ebfdebce1c6415bcf1e3e466bc97643\": container with ID starting with eb6450cada51c4a4df645624857de9493ebfdebce1c6415bcf1e3e466bc97643 not found: ID does not exist" Jul 08 23:06:09.749074 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.749048 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59cb7bf8bc-fzchg"] Jul 08 23:06:09.752934 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:09.752913 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59cb7bf8bc-fzchg"] Jul 08 23:06:10.827780 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:10.827749 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80a886b8-ac40-4557-a4bc-c5f3b1b22fd8" path="/var/lib/kubelet/pods/80a886b8-ac40-4557-a4bc-c5f3b1b22fd8/volumes" Jul 08 23:06:43.200805 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.200773 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-857c96864c-dfpcf"] Jul 08 23:06:43.201264 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.201052 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80a886b8-ac40-4557-a4bc-c5f3b1b22fd8" containerName="console" Jul 08 23:06:43.201264 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.201065 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="80a886b8-ac40-4557-a4bc-c5f3b1b22fd8" containerName="console" Jul 08 23:06:43.201264 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.201114 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="80a886b8-ac40-4557-a4bc-c5f3b1b22fd8" containerName="console" Jul 08 23:06:43.203957 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.203942 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.215731 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.215703 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-857c96864c-dfpcf"] Jul 08 23:06:43.293261 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.293227 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4b19d4a7-63ac-4dec-8337-b09907a31586-console-oauth-config\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.293261 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.293263 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-service-ca\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.293473 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.293309 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-trusted-ca-bundle\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.293473 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.293334 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4b19d4a7-63ac-4dec-8337-b09907a31586-console-serving-cert\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.293473 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.293350 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-console-config\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.293473 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.293376 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-oauth-serving-cert\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.293473 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.293408 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp5qm\" (UniqueName: \"kubernetes.io/projected/4b19d4a7-63ac-4dec-8337-b09907a31586-kube-api-access-zp5qm\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.394447 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.394418 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-trusted-ca-bundle\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.394447 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.394455 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4b19d4a7-63ac-4dec-8337-b09907a31586-console-serving-cert\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.394665 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.394470 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-console-config\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.394665 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.394519 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-oauth-serving-cert\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.394665 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.394569 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp5qm\" (UniqueName: \"kubernetes.io/projected/4b19d4a7-63ac-4dec-8337-b09907a31586-kube-api-access-zp5qm\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.394665 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.394627 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4b19d4a7-63ac-4dec-8337-b09907a31586-console-oauth-config\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.394838 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.394761 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-service-ca\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.395317 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.395290 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-oauth-serving-cert\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.395455 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.395428 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-console-config\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.395517 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.395437 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-trusted-ca-bundle\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.395560 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.395535 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-service-ca\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.397001 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.396974 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4b19d4a7-63ac-4dec-8337-b09907a31586-console-oauth-config\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.397201 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.397184 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4b19d4a7-63ac-4dec-8337-b09907a31586-console-serving-cert\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.401634 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.401615 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp5qm\" (UniqueName: \"kubernetes.io/projected/4b19d4a7-63ac-4dec-8337-b09907a31586-kube-api-access-zp5qm\") pod \"console-857c96864c-dfpcf\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.513383 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.513345 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:43.640827 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.640795 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-857c96864c-dfpcf"] Jul 08 23:06:43.643622 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:06:43.643573 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b19d4a7_63ac_4dec_8337_b09907a31586.slice/crio-e0017f84956e87deb28274ff3ec25bd224d0a0e6fb1bcbf8574fcd63e2a16035 WatchSource:0}: Error finding container e0017f84956e87deb28274ff3ec25bd224d0a0e6fb1bcbf8574fcd63e2a16035: Status 404 returned error can't find the container with id e0017f84956e87deb28274ff3ec25bd224d0a0e6fb1bcbf8574fcd63e2a16035 Jul 08 23:06:43.844170 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.844080 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857c96864c-dfpcf" event={"ID":"4b19d4a7-63ac-4dec-8337-b09907a31586","Type":"ContainerStarted","Data":"a0a8cbc73db045ffab4051325d52798027b4fb70384b3b13730933e6c44e68f4"} Jul 08 23:06:43.844170 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.844144 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857c96864c-dfpcf" event={"ID":"4b19d4a7-63ac-4dec-8337-b09907a31586","Type":"ContainerStarted","Data":"e0017f84956e87deb28274ff3ec25bd224d0a0e6fb1bcbf8574fcd63e2a16035"} Jul 08 23:06:43.862343 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:43.862294 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-857c96864c-dfpcf" podStartSLOduration=0.862275477 podStartE2EDuration="862.275477ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:06:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:06:43.860688821 +0000 UTC m=+225.621183875" watchObservedRunningTime="2026-07-08 23:06:43.862275477 +0000 UTC m=+225.622770588" Jul 08 23:06:53.514064 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:53.514016 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:53.514537 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:53.514084 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:53.518757 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:53.518736 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:53.882222 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:53.882146 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:06:53.926267 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:06:53.926229 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d95948d8c-6rn4z"] Jul 08 23:07:18.952241 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:18.952179 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5d95948d8c-6rn4z" podUID="a911fcc2-f1d6-4805-a089-759127502dbd" containerName="console" containerID="cri-o://4c94ee6f3913f69e58439789b901f47dfc4f38bf2292a113e3a0dd177d2015bb" gracePeriod=15 Jul 08 23:07:19.200888 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.200865 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:07:19.283309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.283266 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2wxc\" (UniqueName: \"kubernetes.io/projected/a911fcc2-f1d6-4805-a089-759127502dbd-kube-api-access-k2wxc\") pod \"a911fcc2-f1d6-4805-a089-759127502dbd\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " Jul 08 23:07:19.283500 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.283323 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-service-ca\") pod \"a911fcc2-f1d6-4805-a089-759127502dbd\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " Jul 08 23:07:19.283500 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.283361 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-oauth-serving-cert\") pod \"a911fcc2-f1d6-4805-a089-759127502dbd\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " Jul 08 23:07:19.283500 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.283393 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-console-config\") pod \"a911fcc2-f1d6-4805-a089-759127502dbd\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " Jul 08 23:07:19.283500 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.283417 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a911fcc2-f1d6-4805-a089-759127502dbd-console-oauth-config\") pod \"a911fcc2-f1d6-4805-a089-759127502dbd\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " Jul 08 23:07:19.283500 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.283464 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-trusted-ca-bundle\") pod \"a911fcc2-f1d6-4805-a089-759127502dbd\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " Jul 08 23:07:19.283500 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.283490 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a911fcc2-f1d6-4805-a089-759127502dbd-console-serving-cert\") pod \"a911fcc2-f1d6-4805-a089-759127502dbd\" (UID: \"a911fcc2-f1d6-4805-a089-759127502dbd\") " Jul 08 23:07:19.283930 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.283898 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-console-config" (OuterVolumeSpecName: "console-config") pod "a911fcc2-f1d6-4805-a089-759127502dbd" (UID: "a911fcc2-f1d6-4805-a089-759127502dbd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:07:19.284060 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.283931 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a911fcc2-f1d6-4805-a089-759127502dbd" (UID: "a911fcc2-f1d6-4805-a089-759127502dbd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:07:19.284123 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.284062 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a911fcc2-f1d6-4805-a089-759127502dbd" (UID: "a911fcc2-f1d6-4805-a089-759127502dbd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:07:19.284175 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.284118 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-service-ca" (OuterVolumeSpecName: "service-ca") pod "a911fcc2-f1d6-4805-a089-759127502dbd" (UID: "a911fcc2-f1d6-4805-a089-759127502dbd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:07:19.285573 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.285539 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a911fcc2-f1d6-4805-a089-759127502dbd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a911fcc2-f1d6-4805-a089-759127502dbd" (UID: "a911fcc2-f1d6-4805-a089-759127502dbd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 23:07:19.285690 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.285625 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a911fcc2-f1d6-4805-a089-759127502dbd-kube-api-access-k2wxc" (OuterVolumeSpecName: "kube-api-access-k2wxc") pod "a911fcc2-f1d6-4805-a089-759127502dbd" (UID: "a911fcc2-f1d6-4805-a089-759127502dbd"). InnerVolumeSpecName "kube-api-access-k2wxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:07:19.285739 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.285692 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a911fcc2-f1d6-4805-a089-759127502dbd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a911fcc2-f1d6-4805-a089-759127502dbd" (UID: "a911fcc2-f1d6-4805-a089-759127502dbd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 23:07:19.384697 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.384660 2548 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-trusted-ca-bundle\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:07:19.384697 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.384691 2548 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a911fcc2-f1d6-4805-a089-759127502dbd-console-serving-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:07:19.384697 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.384702 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2wxc\" (UniqueName: \"kubernetes.io/projected/a911fcc2-f1d6-4805-a089-759127502dbd-kube-api-access-k2wxc\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:07:19.384921 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.384712 2548 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-service-ca\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:07:19.384921 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.384720 2548 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-oauth-serving-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:07:19.384921 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.384729 2548 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a911fcc2-f1d6-4805-a089-759127502dbd-console-config\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:07:19.384921 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.384737 2548 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a911fcc2-f1d6-4805-a089-759127502dbd-console-oauth-config\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:07:19.970225 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.970141 2548 generic.go:356] "Generic (PLEG): container finished" podID="a911fcc2-f1d6-4805-a089-759127502dbd" containerID="4c94ee6f3913f69e58439789b901f47dfc4f38bf2292a113e3a0dd177d2015bb" exitCode=0 Jul 08 23:07:19.970225 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.970182 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d95948d8c-6rn4z" Jul 08 23:07:19.970720 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.970219 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d95948d8c-6rn4z" event={"ID":"a911fcc2-f1d6-4805-a089-759127502dbd","Type":"ContainerDied","Data":"4c94ee6f3913f69e58439789b901f47dfc4f38bf2292a113e3a0dd177d2015bb"} Jul 08 23:07:19.970720 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.970282 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d95948d8c-6rn4z" event={"ID":"a911fcc2-f1d6-4805-a089-759127502dbd","Type":"ContainerDied","Data":"ce9245bbbca1e59113f9645c7d71d93433ab5883a61d1de9879a2c30088286e6"} Jul 08 23:07:19.970720 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.970310 2548 scope.go:117] "RemoveContainer" containerID="4c94ee6f3913f69e58439789b901f47dfc4f38bf2292a113e3a0dd177d2015bb" Jul 08 23:07:19.978498 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.978481 2548 scope.go:117] "RemoveContainer" containerID="4c94ee6f3913f69e58439789b901f47dfc4f38bf2292a113e3a0dd177d2015bb" Jul 08 23:07:19.978786 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:07:19.978766 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c94ee6f3913f69e58439789b901f47dfc4f38bf2292a113e3a0dd177d2015bb\": container with ID starting with 4c94ee6f3913f69e58439789b901f47dfc4f38bf2292a113e3a0dd177d2015bb not found: ID does not exist" containerID="4c94ee6f3913f69e58439789b901f47dfc4f38bf2292a113e3a0dd177d2015bb" Jul 08 23:07:19.978868 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.978791 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c94ee6f3913f69e58439789b901f47dfc4f38bf2292a113e3a0dd177d2015bb"} err="failed to get container status \"4c94ee6f3913f69e58439789b901f47dfc4f38bf2292a113e3a0dd177d2015bb\": rpc error: code = NotFound desc = could not find container \"4c94ee6f3913f69e58439789b901f47dfc4f38bf2292a113e3a0dd177d2015bb\": container with ID starting with 4c94ee6f3913f69e58439789b901f47dfc4f38bf2292a113e3a0dd177d2015bb not found: ID does not exist" Jul 08 23:07:19.989639 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.989614 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d95948d8c-6rn4z"] Jul 08 23:07:19.993859 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:19.993828 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d95948d8c-6rn4z"] Jul 08 23:07:20.827979 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:20.827944 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a911fcc2-f1d6-4805-a089-759127502dbd" path="/var/lib/kubelet/pods/a911fcc2-f1d6-4805-a089-759127502dbd/volumes" Jul 08 23:07:36.751090 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:36.751055 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n"] Jul 08 23:07:36.751511 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:36.751318 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a911fcc2-f1d6-4805-a089-759127502dbd" containerName="console" Jul 08 23:07:36.751511 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:36.751329 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="a911fcc2-f1d6-4805-a089-759127502dbd" containerName="console" Jul 08 23:07:36.751511 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:36.751379 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="a911fcc2-f1d6-4805-a089-759127502dbd" containerName="console" Jul 08 23:07:36.755588 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:36.755572 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n" Jul 08 23:07:36.758290 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:36.758259 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 23:07:36.758416 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:36.758346 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mmj8v\"" Jul 08 23:07:36.759366 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:36.759346 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 23:07:36.763743 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:36.763724 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n"] Jul 08 23:07:36.807740 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:36.807706 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z98c\" (UniqueName: \"kubernetes.io/projected/8611e725-2bcb-4fb5-90ba-af5611abc152-kube-api-access-6z98c\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n\" (UID: \"8611e725-2bcb-4fb5-90ba-af5611abc152\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n" Jul 08 23:07:36.807740 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:36.807748 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8611e725-2bcb-4fb5-90ba-af5611abc152-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n\" (UID: \"8611e725-2bcb-4fb5-90ba-af5611abc152\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n" Jul 08 23:07:36.807940 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:36.807767 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8611e725-2bcb-4fb5-90ba-af5611abc152-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n\" (UID: \"8611e725-2bcb-4fb5-90ba-af5611abc152\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n" Jul 08 23:07:36.909065 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:36.909037 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z98c\" (UniqueName: \"kubernetes.io/projected/8611e725-2bcb-4fb5-90ba-af5611abc152-kube-api-access-6z98c\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n\" (UID: \"8611e725-2bcb-4fb5-90ba-af5611abc152\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n" Jul 08 23:07:36.909232 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:36.909073 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8611e725-2bcb-4fb5-90ba-af5611abc152-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n\" (UID: \"8611e725-2bcb-4fb5-90ba-af5611abc152\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n" Jul 08 23:07:36.909232 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:36.909100 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8611e725-2bcb-4fb5-90ba-af5611abc152-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n\" (UID: \"8611e725-2bcb-4fb5-90ba-af5611abc152\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n" Jul 08 23:07:36.909538 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:36.909518 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8611e725-2bcb-4fb5-90ba-af5611abc152-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n\" (UID: \"8611e725-2bcb-4fb5-90ba-af5611abc152\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n" Jul 08 23:07:36.909538 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:36.909525 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8611e725-2bcb-4fb5-90ba-af5611abc152-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n\" (UID: \"8611e725-2bcb-4fb5-90ba-af5611abc152\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n" Jul 08 23:07:36.917159 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:36.917126 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z98c\" (UniqueName: \"kubernetes.io/projected/8611e725-2bcb-4fb5-90ba-af5611abc152-kube-api-access-6z98c\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n\" (UID: \"8611e725-2bcb-4fb5-90ba-af5611abc152\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n" Jul 08 23:07:37.064943 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:37.064858 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n" Jul 08 23:07:37.190520 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:37.190495 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n"] Jul 08 23:07:37.193135 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:07:37.193110 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8611e725_2bcb_4fb5_90ba_af5611abc152.slice/crio-a2e92228b2f885ee1c7fb4f1c254733b1149b941705cbe43bb0985589d0ba1b5 WatchSource:0}: Error finding container a2e92228b2f885ee1c7fb4f1c254733b1149b941705cbe43bb0985589d0ba1b5: Status 404 returned error can't find the container with id a2e92228b2f885ee1c7fb4f1c254733b1149b941705cbe43bb0985589d0ba1b5 Jul 08 23:07:38.034893 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:38.034857 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n" event={"ID":"8611e725-2bcb-4fb5-90ba-af5611abc152","Type":"ContainerStarted","Data":"a2e92228b2f885ee1c7fb4f1c254733b1149b941705cbe43bb0985589d0ba1b5"} Jul 08 23:07:46.065454 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:46.065421 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n" event={"ID":"8611e725-2bcb-4fb5-90ba-af5611abc152","Type":"ContainerStarted","Data":"313bc7f05addd69419955a9f7a545112b151ee3330165ee570b1ec6b0eced545"} Jul 08 23:07:47.070538 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:47.070502 2548 generic.go:356] "Generic (PLEG): container finished" podID="8611e725-2bcb-4fb5-90ba-af5611abc152" containerID="313bc7f05addd69419955a9f7a545112b151ee3330165ee570b1ec6b0eced545" exitCode=0 Jul 08 23:07:47.070538 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:47.070540 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n" event={"ID":"8611e725-2bcb-4fb5-90ba-af5611abc152","Type":"ContainerDied","Data":"313bc7f05addd69419955a9f7a545112b151ee3330165ee570b1ec6b0eced545"} Jul 08 23:07:50.083980 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:50.083941 2548 generic.go:356] "Generic (PLEG): container finished" podID="8611e725-2bcb-4fb5-90ba-af5611abc152" containerID="f7994fd91ce5d6748c6d30fb689245f54922803bebc7b5b29afd89fb49a0ec48" exitCode=0 Jul 08 23:07:50.084347 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:50.084016 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n" event={"ID":"8611e725-2bcb-4fb5-90ba-af5611abc152","Type":"ContainerDied","Data":"f7994fd91ce5d6748c6d30fb689245f54922803bebc7b5b29afd89fb49a0ec48"} Jul 08 23:07:58.677496 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:58.677470 2548 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 23:07:59.119193 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:59.119162 2548 generic.go:356] "Generic (PLEG): container finished" podID="8611e725-2bcb-4fb5-90ba-af5611abc152" containerID="52181751a149b9bc9dab205c40a14a9d80cfe386a9d42ceb39845b6ebc3e3f2d" exitCode=0 Jul 08 23:07:59.121248 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:07:59.119242 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n" event={"ID":"8611e725-2bcb-4fb5-90ba-af5611abc152","Type":"ContainerDied","Data":"52181751a149b9bc9dab205c40a14a9d80cfe386a9d42ceb39845b6ebc3e3f2d"} Jul 08 23:08:00.249483 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:00.249460 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n" Jul 08 23:08:00.417114 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:00.417003 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6z98c\" (UniqueName: \"kubernetes.io/projected/8611e725-2bcb-4fb5-90ba-af5611abc152-kube-api-access-6z98c\") pod \"8611e725-2bcb-4fb5-90ba-af5611abc152\" (UID: \"8611e725-2bcb-4fb5-90ba-af5611abc152\") " Jul 08 23:08:00.417114 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:00.417107 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8611e725-2bcb-4fb5-90ba-af5611abc152-util\") pod \"8611e725-2bcb-4fb5-90ba-af5611abc152\" (UID: \"8611e725-2bcb-4fb5-90ba-af5611abc152\") " Jul 08 23:08:00.417286 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:00.417207 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8611e725-2bcb-4fb5-90ba-af5611abc152-bundle\") pod \"8611e725-2bcb-4fb5-90ba-af5611abc152\" (UID: \"8611e725-2bcb-4fb5-90ba-af5611abc152\") " Jul 08 23:08:00.417823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:00.417794 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8611e725-2bcb-4fb5-90ba-af5611abc152-bundle" (OuterVolumeSpecName: "bundle") pod "8611e725-2bcb-4fb5-90ba-af5611abc152" (UID: "8611e725-2bcb-4fb5-90ba-af5611abc152"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:08:00.419266 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:00.419237 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8611e725-2bcb-4fb5-90ba-af5611abc152-kube-api-access-6z98c" (OuterVolumeSpecName: "kube-api-access-6z98c") pod "8611e725-2bcb-4fb5-90ba-af5611abc152" (UID: "8611e725-2bcb-4fb5-90ba-af5611abc152"). InnerVolumeSpecName "kube-api-access-6z98c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:08:00.421781 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:00.421761 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8611e725-2bcb-4fb5-90ba-af5611abc152-util" (OuterVolumeSpecName: "util") pod "8611e725-2bcb-4fb5-90ba-af5611abc152" (UID: "8611e725-2bcb-4fb5-90ba-af5611abc152"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:08:00.517793 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:00.517764 2548 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8611e725-2bcb-4fb5-90ba-af5611abc152-util\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:08:00.517793 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:00.517789 2548 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8611e725-2bcb-4fb5-90ba-af5611abc152-bundle\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:08:00.517793 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:00.517797 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z98c\" (UniqueName: \"kubernetes.io/projected/8611e725-2bcb-4fb5-90ba-af5611abc152-kube-api-access-6z98c\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:08:01.128346 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:01.128250 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n" event={"ID":"8611e725-2bcb-4fb5-90ba-af5611abc152","Type":"ContainerDied","Data":"a2e92228b2f885ee1c7fb4f1c254733b1149b941705cbe43bb0985589d0ba1b5"} Jul 08 23:08:01.128346 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:01.128312 2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2e92228b2f885ee1c7fb4f1c254733b1149b941705cbe43bb0985589d0ba1b5" Jul 08 23:08:01.128346 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:01.128263 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f689n" Jul 08 23:08:04.763040 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:04.763002 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-glbbf"] Jul 08 23:08:04.763507 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:04.763370 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8611e725-2bcb-4fb5-90ba-af5611abc152" containerName="pull" Jul 08 23:08:04.763507 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:04.763387 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="8611e725-2bcb-4fb5-90ba-af5611abc152" containerName="pull" Jul 08 23:08:04.763507 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:04.763404 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8611e725-2bcb-4fb5-90ba-af5611abc152" containerName="extract" Jul 08 23:08:04.763507 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:04.763412 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="8611e725-2bcb-4fb5-90ba-af5611abc152" containerName="extract" Jul 08 23:08:04.763507 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:04.763437 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8611e725-2bcb-4fb5-90ba-af5611abc152" containerName="util" Jul 08 23:08:04.763507 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:04.763445 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="8611e725-2bcb-4fb5-90ba-af5611abc152" containerName="util" Jul 08 23:08:04.763507 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:04.763508 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="8611e725-2bcb-4fb5-90ba-af5611abc152" containerName="extract" Jul 08 23:08:04.796067 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:04.796026 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-glbbf"] Jul 08 23:08:04.796221 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:04.796136 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-glbbf" Jul 08 23:08:04.798970 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:04.798948 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 08 23:08:04.799108 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:04.798986 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-x7p4g\"" Jul 08 23:08:04.799108 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:04.799089 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 08 23:08:04.845670 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:04.845639 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2526df09-284c-4edb-934a-c92bf9a612da-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-glbbf\" (UID: \"2526df09-284c-4edb-934a-c92bf9a612da\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-glbbf" Jul 08 23:08:04.845838 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:04.845717 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sx4z\" (UniqueName: \"kubernetes.io/projected/2526df09-284c-4edb-934a-c92bf9a612da-kube-api-access-6sx4z\") pod \"cert-manager-operator-controller-manager-55c444bf86-glbbf\" (UID: \"2526df09-284c-4edb-934a-c92bf9a612da\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-glbbf" Jul 08 23:08:04.947069 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:04.947030 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sx4z\" (UniqueName: \"kubernetes.io/projected/2526df09-284c-4edb-934a-c92bf9a612da-kube-api-access-6sx4z\") pod \"cert-manager-operator-controller-manager-55c444bf86-glbbf\" (UID: \"2526df09-284c-4edb-934a-c92bf9a612da\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-glbbf" Jul 08 23:08:04.947069 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:04.947068 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2526df09-284c-4edb-934a-c92bf9a612da-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-glbbf\" (UID: \"2526df09-284c-4edb-934a-c92bf9a612da\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-glbbf" Jul 08 23:08:04.947432 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:04.947413 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2526df09-284c-4edb-934a-c92bf9a612da-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-glbbf\" (UID: \"2526df09-284c-4edb-934a-c92bf9a612da\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-glbbf" Jul 08 23:08:04.955367 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:04.955337 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sx4z\" (UniqueName: \"kubernetes.io/projected/2526df09-284c-4edb-934a-c92bf9a612da-kube-api-access-6sx4z\") pod \"cert-manager-operator-controller-manager-55c444bf86-glbbf\" (UID: \"2526df09-284c-4edb-934a-c92bf9a612da\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-glbbf" Jul 08 23:08:05.105205 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:05.105113 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-glbbf" Jul 08 23:08:05.251024 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:05.250893 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-glbbf"] Jul 08 23:08:05.253401 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:08:05.253367 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2526df09_284c_4edb_934a_c92bf9a612da.slice/crio-87ddb1df534f43ed26cd90168a0eeb3511299c63f4aec0677152256665ff6d23 WatchSource:0}: Error finding container 87ddb1df534f43ed26cd90168a0eeb3511299c63f4aec0677152256665ff6d23: Status 404 returned error can't find the container with id 87ddb1df534f43ed26cd90168a0eeb3511299c63f4aec0677152256665ff6d23 Jul 08 23:08:05.256007 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:05.255991 2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 23:08:06.146309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:06.146275 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-glbbf" event={"ID":"2526df09-284c-4edb-934a-c92bf9a612da","Type":"ContainerStarted","Data":"87ddb1df534f43ed26cd90168a0eeb3511299c63f4aec0677152256665ff6d23"} Jul 08 23:08:07.151756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:07.151719 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-glbbf" event={"ID":"2526df09-284c-4edb-934a-c92bf9a612da","Type":"ContainerStarted","Data":"68d59a023600e8df021ba7d01f9591ac3ba790af077b5f60ed8405d4598617da"} Jul 08 23:08:07.178729 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:07.178674 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-glbbf" podStartSLOduration=1.416304673 podStartE2EDuration="3.17865142s" totalImagesPullingTime="1.762346747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:08:07.177316617 +0000 UTC m=+308.937811672" watchObservedRunningTime="2026-07-08 23:08:07.17865142 +0000 UTC m=+308.939146476" Jul 08 23:08:08.392259 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:08.392222 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh"] Jul 08 23:08:08.394378 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:08.394359 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh" Jul 08 23:08:08.397043 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:08.397017 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 23:08:08.397166 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:08.397048 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 23:08:08.398074 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:08.398061 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mmj8v\"" Jul 08 23:08:08.404379 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:08.404360 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh"] Jul 08 23:08:08.470606 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:08.470570 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbdr4\" (UniqueName: \"kubernetes.io/projected/efbb1c3f-ed17-4d54-856b-4041d8836f50-kube-api-access-lbdr4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh\" (UID: \"efbb1c3f-ed17-4d54-856b-4041d8836f50\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh" Jul 08 23:08:08.470774 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:08.470627 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efbb1c3f-ed17-4d54-856b-4041d8836f50-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh\" (UID: \"efbb1c3f-ed17-4d54-856b-4041d8836f50\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh" Jul 08 23:08:08.470774 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:08.470661 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efbb1c3f-ed17-4d54-856b-4041d8836f50-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh\" (UID: \"efbb1c3f-ed17-4d54-856b-4041d8836f50\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh" Jul 08 23:08:08.571404 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:08.571366 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbdr4\" (UniqueName: \"kubernetes.io/projected/efbb1c3f-ed17-4d54-856b-4041d8836f50-kube-api-access-lbdr4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh\" (UID: \"efbb1c3f-ed17-4d54-856b-4041d8836f50\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh" Jul 08 23:08:08.571404 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:08.571407 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efbb1c3f-ed17-4d54-856b-4041d8836f50-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh\" (UID: \"efbb1c3f-ed17-4d54-856b-4041d8836f50\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh" Jul 08 23:08:08.571640 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:08.571447 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efbb1c3f-ed17-4d54-856b-4041d8836f50-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh\" (UID: \"efbb1c3f-ed17-4d54-856b-4041d8836f50\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh" Jul 08 23:08:08.571831 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:08.571812 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efbb1c3f-ed17-4d54-856b-4041d8836f50-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh\" (UID: \"efbb1c3f-ed17-4d54-856b-4041d8836f50\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh" Jul 08 23:08:08.571874 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:08.571856 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efbb1c3f-ed17-4d54-856b-4041d8836f50-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh\" (UID: \"efbb1c3f-ed17-4d54-856b-4041d8836f50\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh" Jul 08 23:08:08.580643 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:08.580621 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbdr4\" (UniqueName: \"kubernetes.io/projected/efbb1c3f-ed17-4d54-856b-4041d8836f50-kube-api-access-lbdr4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh\" (UID: \"efbb1c3f-ed17-4d54-856b-4041d8836f50\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh" Jul 08 23:08:08.704105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:08.704022 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh" Jul 08 23:08:08.830510 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:08.830471 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh"] Jul 08 23:08:09.162871 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:09.162830 2548 generic.go:356] "Generic (PLEG): container finished" podID="efbb1c3f-ed17-4d54-856b-4041d8836f50" containerID="ca1340505fdbcc4b9c68e607ac8994c033dd1acd98dea91080df9c1c694582a9" exitCode=0 Jul 08 23:08:09.163044 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:09.162920 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh" event={"ID":"efbb1c3f-ed17-4d54-856b-4041d8836f50","Type":"ContainerDied","Data":"ca1340505fdbcc4b9c68e607ac8994c033dd1acd98dea91080df9c1c694582a9"} Jul 08 23:08:09.163044 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:09.162996 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh" event={"ID":"efbb1c3f-ed17-4d54-856b-4041d8836f50","Type":"ContainerStarted","Data":"e018ca0b1c253f5164cf117d618e0eef985128b4691cd4766eed2779324333fa"} Jul 08 23:08:10.346138 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:10.346106 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-w249h"] Jul 08 23:08:10.348524 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:10.348502 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-w249h" Jul 08 23:08:10.350773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:10.350727 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-2flkc\"" Jul 08 23:08:10.351024 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:10.351002 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 23:08:10.351786 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:10.351767 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 23:08:10.356572 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:10.356315 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-w249h"] Jul 08 23:08:10.382331 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:10.382297 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b0352b4c-388d-42a8-a4d3-ea37f0136899-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-w249h\" (UID: \"b0352b4c-388d-42a8-a4d3-ea37f0136899\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w249h" Jul 08 23:08:10.382489 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:10.382384 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mm9c\" (UniqueName: \"kubernetes.io/projected/b0352b4c-388d-42a8-a4d3-ea37f0136899-kube-api-access-6mm9c\") pod \"cert-manager-webhook-6486ffd997-w249h\" (UID: \"b0352b4c-388d-42a8-a4d3-ea37f0136899\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w249h" Jul 08 23:08:10.482814 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:10.482773 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mm9c\" (UniqueName: \"kubernetes.io/projected/b0352b4c-388d-42a8-a4d3-ea37f0136899-kube-api-access-6mm9c\") pod \"cert-manager-webhook-6486ffd997-w249h\" (UID: \"b0352b4c-388d-42a8-a4d3-ea37f0136899\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w249h" Jul 08 23:08:10.482991 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:10.482832 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b0352b4c-388d-42a8-a4d3-ea37f0136899-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-w249h\" (UID: \"b0352b4c-388d-42a8-a4d3-ea37f0136899\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w249h" Jul 08 23:08:10.490804 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:10.490777 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b0352b4c-388d-42a8-a4d3-ea37f0136899-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-w249h\" (UID: \"b0352b4c-388d-42a8-a4d3-ea37f0136899\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w249h" Jul 08 23:08:10.490947 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:10.490806 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mm9c\" (UniqueName: \"kubernetes.io/projected/b0352b4c-388d-42a8-a4d3-ea37f0136899-kube-api-access-6mm9c\") pod \"cert-manager-webhook-6486ffd997-w249h\" (UID: \"b0352b4c-388d-42a8-a4d3-ea37f0136899\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w249h" Jul 08 23:08:10.667813 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:10.667737 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-w249h" Jul 08 23:08:11.201109 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:11.201088 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-w249h"] Jul 08 23:08:11.203298 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:08:11.203276 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0352b4c_388d_42a8_a4d3_ea37f0136899.slice/crio-b544a296e3c671307b3de5175d882c5b62738e68f350d4b97f7c05b83cff2c3c WatchSource:0}: Error finding container b544a296e3c671307b3de5175d882c5b62738e68f350d4b97f7c05b83cff2c3c: Status 404 returned error can't find the container with id b544a296e3c671307b3de5175d882c5b62738e68f350d4b97f7c05b83cff2c3c Jul 08 23:08:12.179799 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:12.179766 2548 generic.go:356] "Generic (PLEG): container finished" podID="efbb1c3f-ed17-4d54-856b-4041d8836f50" containerID="4bd163a7b6211cb59976eb318c5c290f2f876f02d8ff5f5dbd8e04263e3d1767" exitCode=0 Jul 08 23:08:12.180258 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:12.179855 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh" event={"ID":"efbb1c3f-ed17-4d54-856b-4041d8836f50","Type":"ContainerDied","Data":"4bd163a7b6211cb59976eb318c5c290f2f876f02d8ff5f5dbd8e04263e3d1767"} Jul 08 23:08:12.181536 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:12.181401 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-w249h" event={"ID":"b0352b4c-388d-42a8-a4d3-ea37f0136899","Type":"ContainerStarted","Data":"b544a296e3c671307b3de5175d882c5b62738e68f350d4b97f7c05b83cff2c3c"} Jul 08 23:08:13.188116 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:13.188077 2548 generic.go:356] "Generic (PLEG): container finished" podID="efbb1c3f-ed17-4d54-856b-4041d8836f50" containerID="a64ff2a219118779cd294b21e7a68249f6cf73453f4b98ba3d0acfc4382657c8" exitCode=0 Jul 08 23:08:13.188521 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:13.188162 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh" event={"ID":"efbb1c3f-ed17-4d54-856b-4041d8836f50","Type":"ContainerDied","Data":"a64ff2a219118779cd294b21e7a68249f6cf73453f4b98ba3d0acfc4382657c8"} Jul 08 23:08:14.193873 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:14.193828 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-w249h" event={"ID":"b0352b4c-388d-42a8-a4d3-ea37f0136899","Type":"ContainerStarted","Data":"5c5103e900cd749280b042611f66ec642a928f731c11fc172b8ef556f477b9d3"} Jul 08 23:08:14.194275 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:14.194099 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-w249h" Jul 08 23:08:14.210288 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:14.210233 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-w249h" podStartSLOduration=1.376437851 podStartE2EDuration="4.210217851s" totalImagesPullingTime="2.83378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:08:14.208282292 +0000 UTC m=+315.968777348" watchObservedRunningTime="2026-07-08 23:08:14.210217851 +0000 UTC m=+315.970712904" Jul 08 23:08:14.318277 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:14.318254 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh" Jul 08 23:08:14.410486 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:14.410450 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efbb1c3f-ed17-4d54-856b-4041d8836f50-bundle\") pod \"efbb1c3f-ed17-4d54-856b-4041d8836f50\" (UID: \"efbb1c3f-ed17-4d54-856b-4041d8836f50\") " Jul 08 23:08:14.410486 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:14.410480 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efbb1c3f-ed17-4d54-856b-4041d8836f50-util\") pod \"efbb1c3f-ed17-4d54-856b-4041d8836f50\" (UID: \"efbb1c3f-ed17-4d54-856b-4041d8836f50\") " Jul 08 23:08:14.410712 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:14.410506 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbdr4\" (UniqueName: \"kubernetes.io/projected/efbb1c3f-ed17-4d54-856b-4041d8836f50-kube-api-access-lbdr4\") pod \"efbb1c3f-ed17-4d54-856b-4041d8836f50\" (UID: \"efbb1c3f-ed17-4d54-856b-4041d8836f50\") " Jul 08 23:08:14.410934 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:14.410903 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efbb1c3f-ed17-4d54-856b-4041d8836f50-bundle" (OuterVolumeSpecName: "bundle") pod "efbb1c3f-ed17-4d54-856b-4041d8836f50" (UID: "efbb1c3f-ed17-4d54-856b-4041d8836f50"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:08:14.412730 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:14.412693 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efbb1c3f-ed17-4d54-856b-4041d8836f50-kube-api-access-lbdr4" (OuterVolumeSpecName: "kube-api-access-lbdr4") pod "efbb1c3f-ed17-4d54-856b-4041d8836f50" (UID: "efbb1c3f-ed17-4d54-856b-4041d8836f50"). InnerVolumeSpecName "kube-api-access-lbdr4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:08:14.440151 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:14.440121 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efbb1c3f-ed17-4d54-856b-4041d8836f50-util" (OuterVolumeSpecName: "util") pod "efbb1c3f-ed17-4d54-856b-4041d8836f50" (UID: "efbb1c3f-ed17-4d54-856b-4041d8836f50"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:08:14.511572 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:14.511543 2548 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efbb1c3f-ed17-4d54-856b-4041d8836f50-bundle\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:08:14.511572 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:14.511568 2548 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efbb1c3f-ed17-4d54-856b-4041d8836f50-util\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:08:14.511572 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:14.511577 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbdr4\" (UniqueName: \"kubernetes.io/projected/efbb1c3f-ed17-4d54-856b-4041d8836f50-kube-api-access-lbdr4\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:08:15.199314 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:15.199191 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh" event={"ID":"efbb1c3f-ed17-4d54-856b-4041d8836f50","Type":"ContainerDied","Data":"e018ca0b1c253f5164cf117d618e0eef985128b4691cd4766eed2779324333fa"} Jul 08 23:08:15.199314 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:15.199233 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f26qdh" Jul 08 23:08:15.199314 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:15.199256 2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e018ca0b1c253f5164cf117d618e0eef985128b4691cd4766eed2779324333fa" Jul 08 23:08:18.237428 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:18.237394 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4dz6k"] Jul 08 23:08:18.237829 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:18.237677 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efbb1c3f-ed17-4d54-856b-4041d8836f50" containerName="util" Jul 08 23:08:18.237829 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:18.237688 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="efbb1c3f-ed17-4d54-856b-4041d8836f50" containerName="util" Jul 08 23:08:18.237829 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:18.237702 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efbb1c3f-ed17-4d54-856b-4041d8836f50" containerName="extract" Jul 08 23:08:18.237829 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:18.237707 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="efbb1c3f-ed17-4d54-856b-4041d8836f50" containerName="extract" Jul 08 23:08:18.237829 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:18.237720 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efbb1c3f-ed17-4d54-856b-4041d8836f50" containerName="pull" Jul 08 23:08:18.237829 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:18.237725 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="efbb1c3f-ed17-4d54-856b-4041d8836f50" containerName="pull" Jul 08 23:08:18.237829 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:18.237770 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="efbb1c3f-ed17-4d54-856b-4041d8836f50" containerName="extract" Jul 08 23:08:18.241578 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:18.241557 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4dz6k" Jul 08 23:08:18.243893 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:18.243876 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-hcpqt\"" Jul 08 23:08:18.249650 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:18.249624 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4dz6k"] Jul 08 23:08:18.340019 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:18.339985 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpvnx\" (UniqueName: \"kubernetes.io/projected/006809cb-05db-41be-8714-36a675205aa3-kube-api-access-jpvnx\") pod \"cert-manager-55b4d954d5-4dz6k\" (UID: \"006809cb-05db-41be-8714-36a675205aa3\") " pod="cert-manager/cert-manager-55b4d954d5-4dz6k" Jul 08 23:08:18.340019 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:18.340027 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/006809cb-05db-41be-8714-36a675205aa3-bound-sa-token\") pod \"cert-manager-55b4d954d5-4dz6k\" (UID: \"006809cb-05db-41be-8714-36a675205aa3\") " pod="cert-manager/cert-manager-55b4d954d5-4dz6k" Jul 08 23:08:18.441319 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:18.441284 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpvnx\" (UniqueName: \"kubernetes.io/projected/006809cb-05db-41be-8714-36a675205aa3-kube-api-access-jpvnx\") pod \"cert-manager-55b4d954d5-4dz6k\" (UID: \"006809cb-05db-41be-8714-36a675205aa3\") " pod="cert-manager/cert-manager-55b4d954d5-4dz6k" Jul 08 23:08:18.441319 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:18.441325 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/006809cb-05db-41be-8714-36a675205aa3-bound-sa-token\") pod \"cert-manager-55b4d954d5-4dz6k\" (UID: \"006809cb-05db-41be-8714-36a675205aa3\") " pod="cert-manager/cert-manager-55b4d954d5-4dz6k" Jul 08 23:08:18.449299 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:18.449272 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/006809cb-05db-41be-8714-36a675205aa3-bound-sa-token\") pod \"cert-manager-55b4d954d5-4dz6k\" (UID: \"006809cb-05db-41be-8714-36a675205aa3\") " pod="cert-manager/cert-manager-55b4d954d5-4dz6k" Jul 08 23:08:18.449479 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:18.449457 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpvnx\" (UniqueName: \"kubernetes.io/projected/006809cb-05db-41be-8714-36a675205aa3-kube-api-access-jpvnx\") pod \"cert-manager-55b4d954d5-4dz6k\" (UID: \"006809cb-05db-41be-8714-36a675205aa3\") " pod="cert-manager/cert-manager-55b4d954d5-4dz6k" Jul 08 23:08:18.551061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:18.550979 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4dz6k" Jul 08 23:08:18.675212 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:18.675185 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4dz6k"] Jul 08 23:08:18.677428 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:08:18.677399 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod006809cb_05db_41be_8714_36a675205aa3.slice/crio-77f3cdd808a9aed333e9f721ecc4b82424d23f8ffbee8c5b7c4dd6d1f6bf13ee WatchSource:0}: Error finding container 77f3cdd808a9aed333e9f721ecc4b82424d23f8ffbee8c5b7c4dd6d1f6bf13ee: Status 404 returned error can't find the container with id 77f3cdd808a9aed333e9f721ecc4b82424d23f8ffbee8c5b7c4dd6d1f6bf13ee Jul 08 23:08:19.217255 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:19.217221 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4dz6k" event={"ID":"006809cb-05db-41be-8714-36a675205aa3","Type":"ContainerStarted","Data":"3f63487c77f828edaa39f76f24754473f620655c84bedc0cb7d1a7f47d9d8ac2"} Jul 08 23:08:19.217448 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:19.217268 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4dz6k" event={"ID":"006809cb-05db-41be-8714-36a675205aa3","Type":"ContainerStarted","Data":"77f3cdd808a9aed333e9f721ecc4b82424d23f8ffbee8c5b7c4dd6d1f6bf13ee"} Jul 08 23:08:19.234043 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:19.233986 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-4dz6k" podStartSLOduration=1.233968856 podStartE2EDuration="1.233968856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:08:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:08:19.231882894 +0000 UTC m=+320.992377962" watchObservedRunningTime="2026-07-08 23:08:19.233968856 +0000 UTC m=+320.994463910" Jul 08 23:08:20.201044 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:20.201013 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-w249h" Jul 08 23:08:22.188487 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:22.188455 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wv9s"] Jul 08 23:08:22.191488 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:22.191473 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wv9s" Jul 08 23:08:22.194119 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:22.194097 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 23:08:22.194558 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:22.194541 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-j2gfb\"" Jul 08 23:08:22.195293 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:22.195273 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 23:08:22.200880 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:22.200860 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wv9s"] Jul 08 23:08:22.271459 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:22.271427 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/792f3943-1813-4ede-9320-6555ae2de4dc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9wv9s\" (UID: \"792f3943-1813-4ede-9320-6555ae2de4dc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wv9s" Jul 08 23:08:22.271622 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:22.271490 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v7bv\" (UniqueName: \"kubernetes.io/projected/792f3943-1813-4ede-9320-6555ae2de4dc-kube-api-access-7v7bv\") pod \"openshift-lws-operator-fd8ccff4c-9wv9s\" (UID: \"792f3943-1813-4ede-9320-6555ae2de4dc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wv9s" Jul 08 23:08:22.372901 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:22.372863 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/792f3943-1813-4ede-9320-6555ae2de4dc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9wv9s\" (UID: \"792f3943-1813-4ede-9320-6555ae2de4dc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wv9s" Jul 08 23:08:22.373078 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:22.372937 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v7bv\" (UniqueName: \"kubernetes.io/projected/792f3943-1813-4ede-9320-6555ae2de4dc-kube-api-access-7v7bv\") pod \"openshift-lws-operator-fd8ccff4c-9wv9s\" (UID: \"792f3943-1813-4ede-9320-6555ae2de4dc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wv9s" Jul 08 23:08:22.373329 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:22.373308 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/792f3943-1813-4ede-9320-6555ae2de4dc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9wv9s\" (UID: \"792f3943-1813-4ede-9320-6555ae2de4dc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wv9s" Jul 08 23:08:22.381098 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:22.381065 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v7bv\" (UniqueName: \"kubernetes.io/projected/792f3943-1813-4ede-9320-6555ae2de4dc-kube-api-access-7v7bv\") pod \"openshift-lws-operator-fd8ccff4c-9wv9s\" (UID: \"792f3943-1813-4ede-9320-6555ae2de4dc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wv9s" Jul 08 23:08:22.501245 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:22.501213 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wv9s" Jul 08 23:08:22.624739 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:22.624714 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wv9s"] Jul 08 23:08:22.627159 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:08:22.627127 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod792f3943_1813_4ede_9320_6555ae2de4dc.slice/crio-6f64aa05c59f0854790bd7f6ab6717713ac50b858436502c7be7020160d41ee4 WatchSource:0}: Error finding container 6f64aa05c59f0854790bd7f6ab6717713ac50b858436502c7be7020160d41ee4: Status 404 returned error can't find the container with id 6f64aa05c59f0854790bd7f6ab6717713ac50b858436502c7be7020160d41ee4 Jul 08 23:08:23.233853 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:23.233824 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wv9s" event={"ID":"792f3943-1813-4ede-9320-6555ae2de4dc","Type":"ContainerStarted","Data":"6f64aa05c59f0854790bd7f6ab6717713ac50b858436502c7be7020160d41ee4"} Jul 08 23:08:25.244905 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:25.244861 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wv9s" event={"ID":"792f3943-1813-4ede-9320-6555ae2de4dc","Type":"ContainerStarted","Data":"5ec714ac2ea12a57848c356b581286dada081dbb8482e4cfc2b6633b0e9cdfb0"} Jul 08 23:08:25.260347 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:25.260295 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wv9s" podStartSLOduration=1.605385251 podStartE2EDuration="3.260278781s" totalImagesPullingTime="1.65489353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:08:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:08:25.259171881 +0000 UTC m=+327.019666935" watchObservedRunningTime="2026-07-08 23:08:25.260278781 +0000 UTC m=+327.020773836" Jul 08 23:08:28.049735 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:28.049700 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p"] Jul 08 23:08:28.067985 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:28.067956 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p"] Jul 08 23:08:28.068133 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:28.068068 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p" Jul 08 23:08:28.070865 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:28.070842 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 23:08:28.070975 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:28.070842 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 23:08:28.071836 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:28.071817 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mmj8v\"" Jul 08 23:08:28.126912 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:28.126875 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6597ef78-0f77-44b2-b2b8-534e21f8b8a5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p\" (UID: \"6597ef78-0f77-44b2-b2b8-534e21f8b8a5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p" Jul 08 23:08:28.127056 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:28.126980 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6597ef78-0f77-44b2-b2b8-534e21f8b8a5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p\" (UID: \"6597ef78-0f77-44b2-b2b8-534e21f8b8a5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p" Jul 08 23:08:28.127105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:28.127068 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqd2q\" (UniqueName: \"kubernetes.io/projected/6597ef78-0f77-44b2-b2b8-534e21f8b8a5-kube-api-access-tqd2q\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p\" (UID: \"6597ef78-0f77-44b2-b2b8-534e21f8b8a5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p" Jul 08 23:08:28.228452 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:28.228411 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqd2q\" (UniqueName: \"kubernetes.io/projected/6597ef78-0f77-44b2-b2b8-534e21f8b8a5-kube-api-access-tqd2q\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p\" (UID: \"6597ef78-0f77-44b2-b2b8-534e21f8b8a5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p" Jul 08 23:08:28.228616 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:28.228464 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6597ef78-0f77-44b2-b2b8-534e21f8b8a5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p\" (UID: \"6597ef78-0f77-44b2-b2b8-534e21f8b8a5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p" Jul 08 23:08:28.228616 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:28.228496 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6597ef78-0f77-44b2-b2b8-534e21f8b8a5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p\" (UID: \"6597ef78-0f77-44b2-b2b8-534e21f8b8a5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p" Jul 08 23:08:28.228829 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:28.228813 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6597ef78-0f77-44b2-b2b8-534e21f8b8a5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p\" (UID: \"6597ef78-0f77-44b2-b2b8-534e21f8b8a5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p" Jul 08 23:08:28.228907 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:28.228885 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6597ef78-0f77-44b2-b2b8-534e21f8b8a5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p\" (UID: \"6597ef78-0f77-44b2-b2b8-534e21f8b8a5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p" Jul 08 23:08:28.236749 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:28.236718 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqd2q\" (UniqueName: \"kubernetes.io/projected/6597ef78-0f77-44b2-b2b8-534e21f8b8a5-kube-api-access-tqd2q\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p\" (UID: \"6597ef78-0f77-44b2-b2b8-534e21f8b8a5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p" Jul 08 23:08:28.377269 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:28.377180 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p" Jul 08 23:08:28.517618 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:28.517572 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p"] Jul 08 23:08:28.520138 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:08:28.520113 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6597ef78_0f77_44b2_b2b8_534e21f8b8a5.slice/crio-013a0996f4ce887b64f2ee38cecb38453f0ec6d1905c201b60fff998dd0a44f3 WatchSource:0}: Error finding container 013a0996f4ce887b64f2ee38cecb38453f0ec6d1905c201b60fff998dd0a44f3: Status 404 returned error can't find the container with id 013a0996f4ce887b64f2ee38cecb38453f0ec6d1905c201b60fff998dd0a44f3 Jul 08 23:08:29.264411 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:29.264378 2548 generic.go:356] "Generic (PLEG): container finished" podID="6597ef78-0f77-44b2-b2b8-534e21f8b8a5" containerID="a27fd54b5e52198b266e09aaafc236c88aaeba31d46958db97815d44a9d733bd" exitCode=0 Jul 08 23:08:29.264823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:29.264470 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p" event={"ID":"6597ef78-0f77-44b2-b2b8-534e21f8b8a5","Type":"ContainerDied","Data":"a27fd54b5e52198b266e09aaafc236c88aaeba31d46958db97815d44a9d733bd"} Jul 08 23:08:29.264823 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:29.264527 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p" event={"ID":"6597ef78-0f77-44b2-b2b8-534e21f8b8a5","Type":"ContainerStarted","Data":"013a0996f4ce887b64f2ee38cecb38453f0ec6d1905c201b60fff998dd0a44f3"} Jul 08 23:08:30.270551 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:30.270516 2548 generic.go:356] "Generic (PLEG): container finished" podID="6597ef78-0f77-44b2-b2b8-534e21f8b8a5" containerID="f886d74d65e920b4de68eeb354b239d36131b0db83a191f2f07999122d3d4a52" exitCode=0 Jul 08 23:08:30.270980 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:30.270621 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p" event={"ID":"6597ef78-0f77-44b2-b2b8-534e21f8b8a5","Type":"ContainerDied","Data":"f886d74d65e920b4de68eeb354b239d36131b0db83a191f2f07999122d3d4a52"} Jul 08 23:08:31.276371 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:31.276337 2548 generic.go:356] "Generic (PLEG): container finished" podID="6597ef78-0f77-44b2-b2b8-534e21f8b8a5" containerID="b5b9023f3a017cbfacfd614548cb97eb7ddf89f85c7b670faf1b3d72a35cadd7" exitCode=0 Jul 08 23:08:31.276742 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:31.276416 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p" event={"ID":"6597ef78-0f77-44b2-b2b8-534e21f8b8a5","Type":"ContainerDied","Data":"b5b9023f3a017cbfacfd614548cb97eb7ddf89f85c7b670faf1b3d72a35cadd7"} Jul 08 23:08:32.404311 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:32.404283 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p" Jul 08 23:08:32.467190 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:32.467160 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6597ef78-0f77-44b2-b2b8-534e21f8b8a5-bundle\") pod \"6597ef78-0f77-44b2-b2b8-534e21f8b8a5\" (UID: \"6597ef78-0f77-44b2-b2b8-534e21f8b8a5\") " Jul 08 23:08:32.467342 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:32.467215 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6597ef78-0f77-44b2-b2b8-534e21f8b8a5-util\") pod \"6597ef78-0f77-44b2-b2b8-534e21f8b8a5\" (UID: \"6597ef78-0f77-44b2-b2b8-534e21f8b8a5\") " Jul 08 23:08:32.467342 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:32.467252 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqd2q\" (UniqueName: \"kubernetes.io/projected/6597ef78-0f77-44b2-b2b8-534e21f8b8a5-kube-api-access-tqd2q\") pod \"6597ef78-0f77-44b2-b2b8-534e21f8b8a5\" (UID: \"6597ef78-0f77-44b2-b2b8-534e21f8b8a5\") " Jul 08 23:08:32.467895 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:32.467860 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6597ef78-0f77-44b2-b2b8-534e21f8b8a5-bundle" (OuterVolumeSpecName: "bundle") pod "6597ef78-0f77-44b2-b2b8-534e21f8b8a5" (UID: "6597ef78-0f77-44b2-b2b8-534e21f8b8a5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:08:32.469316 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:32.469294 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6597ef78-0f77-44b2-b2b8-534e21f8b8a5-kube-api-access-tqd2q" (OuterVolumeSpecName: "kube-api-access-tqd2q") pod "6597ef78-0f77-44b2-b2b8-534e21f8b8a5" (UID: "6597ef78-0f77-44b2-b2b8-534e21f8b8a5"). InnerVolumeSpecName "kube-api-access-tqd2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:08:32.472674 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:32.472654 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6597ef78-0f77-44b2-b2b8-534e21f8b8a5-util" (OuterVolumeSpecName: "util") pod "6597ef78-0f77-44b2-b2b8-534e21f8b8a5" (UID: "6597ef78-0f77-44b2-b2b8-534e21f8b8a5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:08:32.567863 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:32.567777 2548 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6597ef78-0f77-44b2-b2b8-534e21f8b8a5-bundle\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:08:32.567863 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:32.567804 2548 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6597ef78-0f77-44b2-b2b8-534e21f8b8a5-util\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:08:32.567863 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:32.567813 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqd2q\" (UniqueName: \"kubernetes.io/projected/6597ef78-0f77-44b2-b2b8-534e21f8b8a5-kube-api-access-tqd2q\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:08:33.286107 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:33.286074 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p" Jul 08 23:08:33.286107 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:33.286081 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af87r8p" event={"ID":"6597ef78-0f77-44b2-b2b8-534e21f8b8a5","Type":"ContainerDied","Data":"013a0996f4ce887b64f2ee38cecb38453f0ec6d1905c201b60fff998dd0a44f3"} Jul 08 23:08:33.286312 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:33.286130 2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="013a0996f4ce887b64f2ee38cecb38453f0ec6d1905c201b60fff998dd0a44f3" Jul 08 23:08:40.615079 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.615049 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-65f9849997-sg85f"] Jul 08 23:08:40.615460 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.615355 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6597ef78-0f77-44b2-b2b8-534e21f8b8a5" containerName="pull" Jul 08 23:08:40.615460 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.615367 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="6597ef78-0f77-44b2-b2b8-534e21f8b8a5" containerName="pull" Jul 08 23:08:40.615460 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.615384 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6597ef78-0f77-44b2-b2b8-534e21f8b8a5" containerName="util" Jul 08 23:08:40.615460 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.615389 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="6597ef78-0f77-44b2-b2b8-534e21f8b8a5" containerName="util" Jul 08 23:08:40.615460 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.615398 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6597ef78-0f77-44b2-b2b8-534e21f8b8a5" containerName="extract" Jul 08 23:08:40.615460 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.615404 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="6597ef78-0f77-44b2-b2b8-534e21f8b8a5" containerName="extract" Jul 08 23:08:40.615460 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.615446 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="6597ef78-0f77-44b2-b2b8-534e21f8b8a5" containerName="extract" Jul 08 23:08:40.620807 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.620791 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65f9849997-sg85f" Jul 08 23:08:40.623468 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.623445 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 23:08:40.624583 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.624568 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-r6445\"" Jul 08 23:08:40.624583 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.624577 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 23:08:40.624758 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.624619 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 23:08:40.627684 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.627652 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65f9849997-sg85f"] Jul 08 23:08:40.731337 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.731303 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v95xd\" (UniqueName: \"kubernetes.io/projected/3096021c-13fa-4c6d-90f4-d248ddc3403f-kube-api-access-v95xd\") pod \"lws-controller-manager-65f9849997-sg85f\" (UID: \"3096021c-13fa-4c6d-90f4-d248ddc3403f\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-sg85f" Jul 08 23:08:40.731337 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.731342 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3096021c-13fa-4c6d-90f4-d248ddc3403f-cert\") pod \"lws-controller-manager-65f9849997-sg85f\" (UID: \"3096021c-13fa-4c6d-90f4-d248ddc3403f\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-sg85f" Jul 08 23:08:40.731554 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.731372 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3096021c-13fa-4c6d-90f4-d248ddc3403f-manager-config\") pod \"lws-controller-manager-65f9849997-sg85f\" (UID: \"3096021c-13fa-4c6d-90f4-d248ddc3403f\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-sg85f" Jul 08 23:08:40.731554 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.731435 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3096021c-13fa-4c6d-90f4-d248ddc3403f-metrics-cert\") pod \"lws-controller-manager-65f9849997-sg85f\" (UID: \"3096021c-13fa-4c6d-90f4-d248ddc3403f\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-sg85f" Jul 08 23:08:40.832487 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.832459 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3096021c-13fa-4c6d-90f4-d248ddc3403f-metrics-cert\") pod \"lws-controller-manager-65f9849997-sg85f\" (UID: \"3096021c-13fa-4c6d-90f4-d248ddc3403f\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-sg85f" Jul 08 23:08:40.832663 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.832514 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v95xd\" (UniqueName: \"kubernetes.io/projected/3096021c-13fa-4c6d-90f4-d248ddc3403f-kube-api-access-v95xd\") pod \"lws-controller-manager-65f9849997-sg85f\" (UID: \"3096021c-13fa-4c6d-90f4-d248ddc3403f\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-sg85f" Jul 08 23:08:40.832724 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.832698 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3096021c-13fa-4c6d-90f4-d248ddc3403f-cert\") pod \"lws-controller-manager-65f9849997-sg85f\" (UID: \"3096021c-13fa-4c6d-90f4-d248ddc3403f\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-sg85f" Jul 08 23:08:40.832764 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.832725 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3096021c-13fa-4c6d-90f4-d248ddc3403f-manager-config\") pod \"lws-controller-manager-65f9849997-sg85f\" (UID: \"3096021c-13fa-4c6d-90f4-d248ddc3403f\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-sg85f" Jul 08 23:08:40.833256 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.833231 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3096021c-13fa-4c6d-90f4-d248ddc3403f-manager-config\") pod \"lws-controller-manager-65f9849997-sg85f\" (UID: \"3096021c-13fa-4c6d-90f4-d248ddc3403f\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-sg85f" Jul 08 23:08:40.834981 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.834960 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3096021c-13fa-4c6d-90f4-d248ddc3403f-metrics-cert\") pod \"lws-controller-manager-65f9849997-sg85f\" (UID: \"3096021c-13fa-4c6d-90f4-d248ddc3403f\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-sg85f" Jul 08 23:08:40.835090 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.835015 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3096021c-13fa-4c6d-90f4-d248ddc3403f-cert\") pod \"lws-controller-manager-65f9849997-sg85f\" (UID: \"3096021c-13fa-4c6d-90f4-d248ddc3403f\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-sg85f" Jul 08 23:08:40.840983 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.840922 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v95xd\" (UniqueName: \"kubernetes.io/projected/3096021c-13fa-4c6d-90f4-d248ddc3403f-kube-api-access-v95xd\") pod \"lws-controller-manager-65f9849997-sg85f\" (UID: \"3096021c-13fa-4c6d-90f4-d248ddc3403f\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-sg85f" Jul 08 23:08:40.930915 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:40.930831 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65f9849997-sg85f" Jul 08 23:08:41.057981 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:41.057954 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65f9849997-sg85f"] Jul 08 23:08:41.060387 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:08:41.060359 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3096021c_13fa_4c6d_90f4_d248ddc3403f.slice/crio-5f38d280030869d66253c4d47f4432e2fbafc5feacf68e703fbf71486c4527d8 WatchSource:0}: Error finding container 5f38d280030869d66253c4d47f4432e2fbafc5feacf68e703fbf71486c4527d8: Status 404 returned error can't find the container with id 5f38d280030869d66253c4d47f4432e2fbafc5feacf68e703fbf71486c4527d8 Jul 08 23:08:41.324792 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:41.324753 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65f9849997-sg85f" event={"ID":"3096021c-13fa-4c6d-90f4-d248ddc3403f","Type":"ContainerStarted","Data":"5f38d280030869d66253c4d47f4432e2fbafc5feacf68e703fbf71486c4527d8"} Jul 08 23:08:43.334655 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:43.334615 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65f9849997-sg85f" event={"ID":"3096021c-13fa-4c6d-90f4-d248ddc3403f","Type":"ContainerStarted","Data":"63feac00a2ec64d4082c95d0079acda740a635cd5f54b36a0b6cc396e747d11e"} Jul 08 23:08:43.335037 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:43.334749 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-65f9849997-sg85f" Jul 08 23:08:43.368795 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:43.368746 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-65f9849997-sg85f" podStartSLOduration=1.53679093 podStartE2EDuration="3.368730743s" totalImagesPullingTime="1.831939813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:08:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:08:43.36783594 +0000 UTC m=+345.128330995" watchObservedRunningTime="2026-07-08 23:08:43.368730743 +0000 UTC m=+345.129225797" Jul 08 23:08:45.559578 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.559545 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt"] Jul 08 23:08:45.604434 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.604395 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt"] Jul 08 23:08:45.604589 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.604511 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" Jul 08 23:08:45.608141 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.608111 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 23:08:45.608967 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.608952 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 23:08:45.609237 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.609222 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 23:08:45.609292 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.609227 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9g6bh\"" Jul 08 23:08:45.611031 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.610752 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 23:08:45.680163 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.680133 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/64fb8ced-2781-4225-ac84-f16f956e4061-manifests\") pod \"opendatahub-operator-controller-manager-ccc7894f9-x75gt\" (UID: \"64fb8ced-2781-4225-ac84-f16f956e4061\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" Jul 08 23:08:45.680336 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.680173 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/64fb8ced-2781-4225-ac84-f16f956e4061-webhook-cert\") pod \"opendatahub-operator-controller-manager-ccc7894f9-x75gt\" (UID: \"64fb8ced-2781-4225-ac84-f16f956e4061\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" Jul 08 23:08:45.680336 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.680207 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/64fb8ced-2781-4225-ac84-f16f956e4061-apiservice-cert\") pod \"opendatahub-operator-controller-manager-ccc7894f9-x75gt\" (UID: \"64fb8ced-2781-4225-ac84-f16f956e4061\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" Jul 08 23:08:45.680336 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.680295 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kskh8\" (UniqueName: \"kubernetes.io/projected/64fb8ced-2781-4225-ac84-f16f956e4061-kube-api-access-kskh8\") pod \"opendatahub-operator-controller-manager-ccc7894f9-x75gt\" (UID: \"64fb8ced-2781-4225-ac84-f16f956e4061\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" Jul 08 23:08:45.680336 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.680331 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64fb8ced-2781-4225-ac84-f16f956e4061-tmp\") pod \"opendatahub-operator-controller-manager-ccc7894f9-x75gt\" (UID: \"64fb8ced-2781-4225-ac84-f16f956e4061\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" Jul 08 23:08:45.781418 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.781376 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/64fb8ced-2781-4225-ac84-f16f956e4061-manifests\") pod \"opendatahub-operator-controller-manager-ccc7894f9-x75gt\" (UID: \"64fb8ced-2781-4225-ac84-f16f956e4061\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" Jul 08 23:08:45.781418 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.781420 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/64fb8ced-2781-4225-ac84-f16f956e4061-webhook-cert\") pod \"opendatahub-operator-controller-manager-ccc7894f9-x75gt\" (UID: \"64fb8ced-2781-4225-ac84-f16f956e4061\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" Jul 08 23:08:45.781699 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.781449 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/64fb8ced-2781-4225-ac84-f16f956e4061-apiservice-cert\") pod \"opendatahub-operator-controller-manager-ccc7894f9-x75gt\" (UID: \"64fb8ced-2781-4225-ac84-f16f956e4061\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" Jul 08 23:08:45.781699 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.781487 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kskh8\" (UniqueName: \"kubernetes.io/projected/64fb8ced-2781-4225-ac84-f16f956e4061-kube-api-access-kskh8\") pod \"opendatahub-operator-controller-manager-ccc7894f9-x75gt\" (UID: \"64fb8ced-2781-4225-ac84-f16f956e4061\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" Jul 08 23:08:45.781699 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.781521 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64fb8ced-2781-4225-ac84-f16f956e4061-tmp\") pod \"opendatahub-operator-controller-manager-ccc7894f9-x75gt\" (UID: \"64fb8ced-2781-4225-ac84-f16f956e4061\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" Jul 08 23:08:45.782023 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.781982 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/64fb8ced-2781-4225-ac84-f16f956e4061-manifests\") pod \"opendatahub-operator-controller-manager-ccc7894f9-x75gt\" (UID: \"64fb8ced-2781-4225-ac84-f16f956e4061\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" Jul 08 23:08:45.784034 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.784004 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64fb8ced-2781-4225-ac84-f16f956e4061-tmp\") pod \"opendatahub-operator-controller-manager-ccc7894f9-x75gt\" (UID: \"64fb8ced-2781-4225-ac84-f16f956e4061\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" Jul 08 23:08:45.784352 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.784331 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/64fb8ced-2781-4225-ac84-f16f956e4061-apiservice-cert\") pod \"opendatahub-operator-controller-manager-ccc7894f9-x75gt\" (UID: \"64fb8ced-2781-4225-ac84-f16f956e4061\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" Jul 08 23:08:45.784462 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.784439 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/64fb8ced-2781-4225-ac84-f16f956e4061-webhook-cert\") pod \"opendatahub-operator-controller-manager-ccc7894f9-x75gt\" (UID: \"64fb8ced-2781-4225-ac84-f16f956e4061\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" Jul 08 23:08:45.789812 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.789739 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kskh8\" (UniqueName: \"kubernetes.io/projected/64fb8ced-2781-4225-ac84-f16f956e4061-kube-api-access-kskh8\") pod \"opendatahub-operator-controller-manager-ccc7894f9-x75gt\" (UID: \"64fb8ced-2781-4225-ac84-f16f956e4061\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" Jul 08 23:08:45.914982 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:45.914864 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" Jul 08 23:08:46.068106 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:46.068081 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt"] Jul 08 23:08:46.070069 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:08:46.070027 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64fb8ced_2781_4225_ac84_f16f956e4061.slice/crio-7049144db026b41f8c527ab660598c044a8d83022384bf337cc39ffcc639c2e6 WatchSource:0}: Error finding container 7049144db026b41f8c527ab660598c044a8d83022384bf337cc39ffcc639c2e6: Status 404 returned error can't find the container with id 7049144db026b41f8c527ab660598c044a8d83022384bf337cc39ffcc639c2e6 Jul 08 23:08:46.347477 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:46.347441 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" event={"ID":"64fb8ced-2781-4225-ac84-f16f956e4061","Type":"ContainerStarted","Data":"7049144db026b41f8c527ab660598c044a8d83022384bf337cc39ffcc639c2e6"} Jul 08 23:08:49.367011 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:49.366923 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" event={"ID":"64fb8ced-2781-4225-ac84-f16f956e4061","Type":"ContainerStarted","Data":"1f2ed318554493720576525dc94857777c0d7fa92e318afbbcb1c0eefe23152d"} Jul 08 23:08:50.372936 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:50.372898 2548 generic.go:356] "Generic (PLEG): container finished" podID="64fb8ced-2781-4225-ac84-f16f956e4061" containerID="1f2ed318554493720576525dc94857777c0d7fa92e318afbbcb1c0eefe23152d" exitCode=0 Jul 08 23:08:50.373356 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:50.372948 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" event={"ID":"64fb8ced-2781-4225-ac84-f16f956e4061","Type":"ContainerDied","Data":"1f2ed318554493720576525dc94857777c0d7fa92e318afbbcb1c0eefe23152d"} Jul 08 23:08:51.381440 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:51.381409 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" event={"ID":"64fb8ced-2781-4225-ac84-f16f956e4061","Type":"ContainerStarted","Data":"86133b0157f140b26860db487e2f272d298f5a0525b227391c2e8efe767fc8f8"} Jul 08 23:08:51.381937 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:51.381520 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" Jul 08 23:08:51.404525 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:51.404470 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" podStartSLOduration=1.8794282820000001 podStartE2EDuration="6.404455533s" totalImagesPullingTime="3.462553125s" totalInitContainerRuntime="1.062474126s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:08:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:08:51.401647092 +0000 UTC m=+353.162142146" watchObservedRunningTime="2026-07-08 23:08:51.404455533 +0000 UTC m=+353.164950586" Jul 08 23:08:54.340647 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:08:54.340589 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-65f9849997-sg85f" Jul 08 23:09:02.387204 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:02.387175 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-x75gt" Jul 08 23:09:22.869923 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:22.869890 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g"] Jul 08 23:09:22.873111 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:22.873092 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:22.875758 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:22.875728 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 23:09:22.875905 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:22.875856 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 23:09:22.875905 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:22.875875 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 23:09:22.876007 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:22.875906 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-5xfx5\"" Jul 08 23:09:22.885306 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:22.885282 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g"] Jul 08 23:09:22.962061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:22.962028 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9"] Jul 08 23:09:22.965362 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:22.965336 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:22.978112 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:22.978083 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9"] Jul 08 23:09:22.991156 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:22.991130 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:22.991313 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:22.991170 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:22.991313 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:22.991201 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:22.991313 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:22.991258 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:22.991472 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:22.991326 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:22.991472 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:22.991353 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:22.991472 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:22.991418 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g692z\" (UniqueName: \"kubernetes.io/projected/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-kube-api-access-g692z\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:22.991472 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:22.991452 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:22.991661 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:22.991497 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.092105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092063 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/104e3c4c-f2de-4b0e-8b82-c69872e92066-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.092298 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092130 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.092298 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092168 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.092298 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092198 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/104e3c4c-f2de-4b0e-8b82-c69872e92066-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.092298 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092228 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/104e3c4c-f2de-4b0e-8b82-c69872e92066-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.092475 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092338 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/104e3c4c-f2de-4b0e-8b82-c69872e92066-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.092475 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092383 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/104e3c4c-f2de-4b0e-8b82-c69872e92066-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.092475 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092461 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g692z\" (UniqueName: \"kubernetes.io/projected/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-kube-api-access-g692z\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.092624 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092502 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlwm8\" (UniqueName: \"kubernetes.io/projected/104e3c4c-f2de-4b0e-8b82-c69872e92066-kube-api-access-wlwm8\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.092624 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092534 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.092624 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092568 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.092624 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092568 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/104e3c4c-f2de-4b0e-8b82-c69872e92066-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.092809 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092653 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.092809 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092653 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.092809 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092691 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.092809 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092739 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.092809 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092771 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.092809 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092803 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.093100 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092842 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/104e3c4c-f2de-4b0e-8b82-c69872e92066-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.093100 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092889 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/104e3c4c-f2de-4b0e-8b82-c69872e92066-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.093100 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.092904 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.093253 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.093151 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.093633 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.093584 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.094991 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.094971 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.095080 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.095060 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.111357 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.111326 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.111486 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.111446 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g692z\" (UniqueName: \"kubernetes.io/projected/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-kube-api-access-g692z\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.184045 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.183957 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:23.194286 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.194259 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/104e3c4c-f2de-4b0e-8b82-c69872e92066-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.194423 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.194292 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/104e3c4c-f2de-4b0e-8b82-c69872e92066-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.194423 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.194313 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/104e3c4c-f2de-4b0e-8b82-c69872e92066-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.194423 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.194349 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/104e3c4c-f2de-4b0e-8b82-c69872e92066-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.194423 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.194378 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/104e3c4c-f2de-4b0e-8b82-c69872e92066-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.194423 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.194412 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/104e3c4c-f2de-4b0e-8b82-c69872e92066-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.194706 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.194439 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/104e3c4c-f2de-4b0e-8b82-c69872e92066-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.194706 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.194478 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlwm8\" (UniqueName: \"kubernetes.io/projected/104e3c4c-f2de-4b0e-8b82-c69872e92066-kube-api-access-wlwm8\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.194706 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.194521 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/104e3c4c-f2de-4b0e-8b82-c69872e92066-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.195082 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.194917 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/104e3c4c-f2de-4b0e-8b82-c69872e92066-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.195082 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.195003 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/104e3c4c-f2de-4b0e-8b82-c69872e92066-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.195082 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.195039 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/104e3c4c-f2de-4b0e-8b82-c69872e92066-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.195296 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.195258 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/104e3c4c-f2de-4b0e-8b82-c69872e92066-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.195922 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.195861 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/104e3c4c-f2de-4b0e-8b82-c69872e92066-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.196948 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.196930 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/104e3c4c-f2de-4b0e-8b82-c69872e92066-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.197374 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.197342 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/104e3c4c-f2de-4b0e-8b82-c69872e92066-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.202706 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.202687 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlwm8\" (UniqueName: \"kubernetes.io/projected/104e3c4c-f2de-4b0e-8b82-c69872e92066-kube-api-access-wlwm8\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.202889 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.202869 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/104e3c4c-f2de-4b0e-8b82-c69872e92066-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4br7p9\" (UID: \"104e3c4c-f2de-4b0e-8b82-c69872e92066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.276981 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.276952 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:23.322398 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.322363 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g"] Jul 08 23:09:23.324093 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:09:23.324063 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec9ac1f6_dfe0_48d6_85ed_4ef80b55528e.slice/crio-ccd57038d6f080e4342d8a7135a24be1d7d0d5267816a9618295a854851cf620 WatchSource:0}: Error finding container ccd57038d6f080e4342d8a7135a24be1d7d0d5267816a9618295a854851cf620: Status 404 returned error can't find the container with id ccd57038d6f080e4342d8a7135a24be1d7d0d5267816a9618295a854851cf620 Jul 08 23:09:23.410250 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.410227 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9"] Jul 08 23:09:23.412127 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:09:23.412101 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod104e3c4c_f2de_4b0e_8b82_c69872e92066.slice/crio-766b6d488552c1a0ad9917895a3797cbdb1aae4e473f77749d4bd7cbdca493aa WatchSource:0}: Error finding container 766b6d488552c1a0ad9917895a3797cbdb1aae4e473f77749d4bd7cbdca493aa: Status 404 returned error can't find the container with id 766b6d488552c1a0ad9917895a3797cbdb1aae4e473f77749d4bd7cbdca493aa Jul 08 23:09:23.523654 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.523623 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" event={"ID":"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e","Type":"ContainerStarted","Data":"ccd57038d6f080e4342d8a7135a24be1d7d0d5267816a9618295a854851cf620"} Jul 08 23:09:23.528965 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:23.528889 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" event={"ID":"104e3c4c-f2de-4b0e-8b82-c69872e92066","Type":"ContainerStarted","Data":"766b6d488552c1a0ad9917895a3797cbdb1aae4e473f77749d4bd7cbdca493aa"} Jul 08 23:09:26.403511 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:26.403468 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:09:26.403845 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:26.403542 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:09:26.403845 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:26.403570 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:09:26.409812 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:26.409787 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:09:26.409889 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:26.409849 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:09:26.409889 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:26.409878 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:09:26.548672 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:26.548634 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" event={"ID":"104e3c4c-f2de-4b0e-8b82-c69872e92066","Type":"ContainerStarted","Data":"d1ed81795720b5947e0bcf1c8afdfe9b49b38cc4c266ca0da59a9166537e535f"} Jul 08 23:09:26.550462 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:26.550432 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" event={"ID":"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e","Type":"ContainerStarted","Data":"390522065a85e5a8aa73b7ce971cfd96c12a2abb63312aff040621b474f0426c"} Jul 08 23:09:26.568325 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:26.568263 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" podStartSLOduration=1.572851012 podStartE2EDuration="4.568222366s" totalImagesPullingTime="2.995371354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:09:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:09:26.56694369 +0000 UTC m=+388.327438745" watchObservedRunningTime="2026-07-08 23:09:26.568222366 +0000 UTC m=+388.328717421" Jul 08 23:09:26.587720 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:26.587660 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" podStartSLOduration=1.510672074 podStartE2EDuration="4.587643274s" totalImagesPullingTime="3.0769712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:09:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:09:26.58498871 +0000 UTC m=+388.345483771" watchObservedRunningTime="2026-07-08 23:09:26.587643274 +0000 UTC m=+388.348138329" Jul 08 23:09:27.184902 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:27.184865 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:27.186334 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:27.186309 2548 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 08 23:09:27.186462 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:27.186389 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" podUID="ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 08 23:09:27.277097 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:27.277062 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:27.282052 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:27.282028 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:27.555864 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:27.555837 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:27.556906 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:27.556888 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4br7p9" Jul 08 23:09:27.603438 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:27.603406 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g"] Jul 08 23:09:28.184802 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:28.184760 2548 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 08 23:09:28.185011 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:28.184838 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" podUID="ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 08 23:09:29.184871 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:29.184834 2548 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 08 23:09:29.185236 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:29.184889 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" podUID="ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 08 23:09:29.565622 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:29.565561 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" podUID="ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e" containerName="istio-proxy" containerID="cri-o://390522065a85e5a8aa73b7ce971cfd96c12a2abb63312aff040621b474f0426c" gracePeriod=30 Jul 08 23:09:34.815232 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.815205 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:34.903988 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.903902 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g692z\" (UniqueName: \"kubernetes.io/projected/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-kube-api-access-g692z\") pod \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " Jul 08 23:09:34.903988 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.903944 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-data\") pod \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " Jul 08 23:09:34.903988 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.903970 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-envoy\") pod \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " Jul 08 23:09:34.903988 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.903989 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-podinfo\") pod \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " Jul 08 23:09:34.904274 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.904027 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-token\") pod \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " Jul 08 23:09:34.904274 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.904060 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istiod-ca-cert\") pod \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " Jul 08 23:09:34.904274 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.904137 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-workload-socket\") pod \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " Jul 08 23:09:34.904274 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.904171 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-workload-certs\") pod \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " Jul 08 23:09:34.904274 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.904196 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-credential-socket\") pod \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\" (UID: \"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e\") " Jul 08 23:09:34.904274 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.904211 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-data" (OuterVolumeSpecName: "istio-data") pod "ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e" (UID: "ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:09:34.904576 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.904394 2548 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-data\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:09:34.904576 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.904442 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e" (UID: "ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:09:34.904720 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.904578 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e" (UID: "ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:09:34.904720 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.904688 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e" (UID: "ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:09:34.904720 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.904699 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e" (UID: "ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:09:34.906410 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.906382 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e" (UID: "ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 23:09:34.906515 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.906436 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e" (UID: "ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:09:34.906570 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.906546 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-kube-api-access-g692z" (OuterVolumeSpecName: "kube-api-access-g692z") pod "ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e" (UID: "ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e"). InnerVolumeSpecName "kube-api-access-g692z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:09:34.906570 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:34.906559 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-token" (OuterVolumeSpecName: "istio-token") pod "ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e" (UID: "ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:09:35.005562 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:35.005518 2548 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istiod-ca-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:09:35.005562 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:35.005554 2548 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-workload-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:09:35.005562 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:35.005563 2548 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-workload-certs\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:09:35.005562 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:35.005571 2548 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-credential-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:09:35.005848 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:35.005580 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g692z\" (UniqueName: \"kubernetes.io/projected/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-kube-api-access-g692z\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:09:35.005848 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:35.005590 2548 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-envoy\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:09:35.005848 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:35.005620 2548 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-podinfo\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:09:35.005848 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:35.005627 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e-istio-token\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:09:35.592506 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:35.592466 2548 generic.go:356] "Generic (PLEG): container finished" podID="ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e" containerID="390522065a85e5a8aa73b7ce971cfd96c12a2abb63312aff040621b474f0426c" exitCode=0 Jul 08 23:09:35.592688 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:35.592520 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" Jul 08 23:09:35.592688 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:35.592538 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" event={"ID":"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e","Type":"ContainerDied","Data":"390522065a85e5a8aa73b7ce971cfd96c12a2abb63312aff040621b474f0426c"} Jul 08 23:09:35.592688 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:35.592592 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g" event={"ID":"ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e","Type":"ContainerDied","Data":"ccd57038d6f080e4342d8a7135a24be1d7d0d5267816a9618295a854851cf620"} Jul 08 23:09:35.592688 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:35.592624 2548 scope.go:117] "RemoveContainer" containerID="390522065a85e5a8aa73b7ce971cfd96c12a2abb63312aff040621b474f0426c" Jul 08 23:09:35.601348 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:35.601328 2548 scope.go:117] "RemoveContainer" containerID="390522065a85e5a8aa73b7ce971cfd96c12a2abb63312aff040621b474f0426c" Jul 08 23:09:35.601608 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:09:35.601579 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"390522065a85e5a8aa73b7ce971cfd96c12a2abb63312aff040621b474f0426c\": container with ID starting with 390522065a85e5a8aa73b7ce971cfd96c12a2abb63312aff040621b474f0426c not found: ID does not exist" containerID="390522065a85e5a8aa73b7ce971cfd96c12a2abb63312aff040621b474f0426c" Jul 08 23:09:35.601665 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:35.601620 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"390522065a85e5a8aa73b7ce971cfd96c12a2abb63312aff040621b474f0426c"} err="failed to get container status \"390522065a85e5a8aa73b7ce971cfd96c12a2abb63312aff040621b474f0426c\": rpc error: code = NotFound desc = could not find container \"390522065a85e5a8aa73b7ce971cfd96c12a2abb63312aff040621b474f0426c\": container with ID starting with 390522065a85e5a8aa73b7ce971cfd96c12a2abb63312aff040621b474f0426c not found: ID does not exist" Jul 08 23:09:35.614748 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:35.614721 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g"] Jul 08 23:09:35.620432 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:35.620405 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-gwh4g"] Jul 08 23:09:36.828208 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:09:36.828177 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e" path="/var/lib/kubelet/pods/ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e/volumes" Jul 08 23:10:06.374014 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:06.373977 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-v7z9d"] Jul 08 23:10:06.374416 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:06.374301 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e" containerName="istio-proxy" Jul 08 23:10:06.374416 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:06.374313 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e" containerName="istio-proxy" Jul 08 23:10:06.374416 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:06.374364 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec9ac1f6-dfe0-48d6-85ed-4ef80b55528e" containerName="istio-proxy" Jul 08 23:10:06.384321 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:06.384295 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-v7z9d" Jul 08 23:10:06.387043 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:06.387017 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 23:10:06.387204 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:06.387083 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-rmrd8\"" Jul 08 23:10:06.387204 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:06.387045 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 23:10:06.387858 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:06.387815 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-v7z9d"] Jul 08 23:10:06.472031 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:06.471993 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkpr9\" (UniqueName: \"kubernetes.io/projected/583cecc3-8c60-4c1d-9c94-8a34a4c31d16-kube-api-access-zkpr9\") pod \"kuadrant-operator-catalog-v7z9d\" (UID: \"583cecc3-8c60-4c1d-9c94-8a34a4c31d16\") " pod="kuadrant-system/kuadrant-operator-catalog-v7z9d" Jul 08 23:10:06.573496 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:06.573460 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkpr9\" (UniqueName: \"kubernetes.io/projected/583cecc3-8c60-4c1d-9c94-8a34a4c31d16-kube-api-access-zkpr9\") pod \"kuadrant-operator-catalog-v7z9d\" (UID: \"583cecc3-8c60-4c1d-9c94-8a34a4c31d16\") " pod="kuadrant-system/kuadrant-operator-catalog-v7z9d" Jul 08 23:10:06.581327 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:06.581300 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkpr9\" (UniqueName: \"kubernetes.io/projected/583cecc3-8c60-4c1d-9c94-8a34a4c31d16-kube-api-access-zkpr9\") pod \"kuadrant-operator-catalog-v7z9d\" (UID: \"583cecc3-8c60-4c1d-9c94-8a34a4c31d16\") " pod="kuadrant-system/kuadrant-operator-catalog-v7z9d" Jul 08 23:10:06.696987 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:06.696905 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-v7z9d" Jul 08 23:10:06.748091 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:06.748060 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-v7z9d"] Jul 08 23:10:06.828972 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:10:06.828941 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod583cecc3_8c60_4c1d_9c94_8a34a4c31d16.slice/crio-e153860a37e21da19b861703a936ba5c76a2fc4acfc2c7f79d893e6b20a77f5f WatchSource:0}: Error finding container e153860a37e21da19b861703a936ba5c76a2fc4acfc2c7f79d893e6b20a77f5f: Status 404 returned error can't find the container with id e153860a37e21da19b861703a936ba5c76a2fc4acfc2c7f79d893e6b20a77f5f Jul 08 23:10:06.829780 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:06.829764 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-v7z9d"] Jul 08 23:10:06.955584 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:06.955508 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r4xs4"] Jul 08 23:10:06.960261 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:06.960243 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r4xs4" Jul 08 23:10:06.966228 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:06.966205 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r4xs4"] Jul 08 23:10:07.079181 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:07.079146 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltvvw\" (UniqueName: \"kubernetes.io/projected/315f4cc0-5159-4ac6-9a39-9d4175af042a-kube-api-access-ltvvw\") pod \"kuadrant-operator-catalog-r4xs4\" (UID: \"315f4cc0-5159-4ac6-9a39-9d4175af042a\") " pod="kuadrant-system/kuadrant-operator-catalog-r4xs4" Jul 08 23:10:07.179827 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:07.179790 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltvvw\" (UniqueName: \"kubernetes.io/projected/315f4cc0-5159-4ac6-9a39-9d4175af042a-kube-api-access-ltvvw\") pod \"kuadrant-operator-catalog-r4xs4\" (UID: \"315f4cc0-5159-4ac6-9a39-9d4175af042a\") " pod="kuadrant-system/kuadrant-operator-catalog-r4xs4" Jul 08 23:10:07.187419 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:07.187396 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltvvw\" (UniqueName: \"kubernetes.io/projected/315f4cc0-5159-4ac6-9a39-9d4175af042a-kube-api-access-ltvvw\") pod \"kuadrant-operator-catalog-r4xs4\" (UID: \"315f4cc0-5159-4ac6-9a39-9d4175af042a\") " pod="kuadrant-system/kuadrant-operator-catalog-r4xs4" Jul 08 23:10:07.270965 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:07.270932 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r4xs4" Jul 08 23:10:07.408485 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:07.408463 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r4xs4"] Jul 08 23:10:07.458502 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:10:07.458449 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod315f4cc0_5159_4ac6_9a39_9d4175af042a.slice/crio-85635c784f4f2fcb9a373907727e18644b17862bbbca97f46e32e5543300e39c WatchSource:0}: Error finding container 85635c784f4f2fcb9a373907727e18644b17862bbbca97f46e32e5543300e39c: Status 404 returned error can't find the container with id 85635c784f4f2fcb9a373907727e18644b17862bbbca97f46e32e5543300e39c Jul 08 23:10:07.743441 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:07.743408 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-v7z9d" event={"ID":"583cecc3-8c60-4c1d-9c94-8a34a4c31d16","Type":"ContainerStarted","Data":"e153860a37e21da19b861703a936ba5c76a2fc4acfc2c7f79d893e6b20a77f5f"} Jul 08 23:10:07.744880 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:07.744847 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r4xs4" event={"ID":"315f4cc0-5159-4ac6-9a39-9d4175af042a","Type":"ContainerStarted","Data":"85635c784f4f2fcb9a373907727e18644b17862bbbca97f46e32e5543300e39c"} Jul 08 23:10:09.758666 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:09.758628 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-v7z9d" event={"ID":"583cecc3-8c60-4c1d-9c94-8a34a4c31d16","Type":"ContainerStarted","Data":"c4fe212a96fdf7ddcece7b3ce8d9299b2f3ac21cc613bcf92c26236da6ef5749"} Jul 08 23:10:09.759177 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:09.758680 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-v7z9d" podUID="583cecc3-8c60-4c1d-9c94-8a34a4c31d16" containerName="registry-server" containerID="cri-o://c4fe212a96fdf7ddcece7b3ce8d9299b2f3ac21cc613bcf92c26236da6ef5749" gracePeriod=2 Jul 08 23:10:09.760439 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:09.760404 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r4xs4" event={"ID":"315f4cc0-5159-4ac6-9a39-9d4175af042a","Type":"ContainerStarted","Data":"429f18eabc37cc09769994598a1f55b48f300b571566c5b1e7c5a573a2b5ebb2"} Jul 08 23:10:09.774205 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:09.774161 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-v7z9d" podStartSLOduration=1.61471638 podStartE2EDuration="3.774147212s" totalImagesPullingTime="2.159430832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:10:09.773264059 +0000 UTC m=+431.533759114" watchObservedRunningTime="2026-07-08 23:10:09.774147212 +0000 UTC m=+431.534642315" Jul 08 23:10:09.787812 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:09.787763 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-r4xs4" podStartSLOduration=2.255056578 podStartE2EDuration="3.787747081s" totalImagesPullingTime="1.532690503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:10:09.787522092 +0000 UTC m=+431.548017144" watchObservedRunningTime="2026-07-08 23:10:09.787747081 +0000 UTC m=+431.548242134" Jul 08 23:10:10.000191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:10.000165 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-v7z9d" Jul 08 23:10:10.105510 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:10.105422 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkpr9\" (UniqueName: \"kubernetes.io/projected/583cecc3-8c60-4c1d-9c94-8a34a4c31d16-kube-api-access-zkpr9\") pod \"583cecc3-8c60-4c1d-9c94-8a34a4c31d16\" (UID: \"583cecc3-8c60-4c1d-9c94-8a34a4c31d16\") " Jul 08 23:10:10.107540 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:10.107506 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/583cecc3-8c60-4c1d-9c94-8a34a4c31d16-kube-api-access-zkpr9" (OuterVolumeSpecName: "kube-api-access-zkpr9") pod "583cecc3-8c60-4c1d-9c94-8a34a4c31d16" (UID: "583cecc3-8c60-4c1d-9c94-8a34a4c31d16"). InnerVolumeSpecName "kube-api-access-zkpr9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:10:10.206219 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:10.206183 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkpr9\" (UniqueName: \"kubernetes.io/projected/583cecc3-8c60-4c1d-9c94-8a34a4c31d16-kube-api-access-zkpr9\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:10:10.765654 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:10.765592 2548 generic.go:356] "Generic (PLEG): container finished" podID="583cecc3-8c60-4c1d-9c94-8a34a4c31d16" containerID="c4fe212a96fdf7ddcece7b3ce8d9299b2f3ac21cc613bcf92c26236da6ef5749" exitCode=0 Jul 08 23:10:10.766130 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:10.765664 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-v7z9d" Jul 08 23:10:10.766130 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:10.765685 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-v7z9d" event={"ID":"583cecc3-8c60-4c1d-9c94-8a34a4c31d16","Type":"ContainerDied","Data":"c4fe212a96fdf7ddcece7b3ce8d9299b2f3ac21cc613bcf92c26236da6ef5749"} Jul 08 23:10:10.766130 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:10.765760 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-v7z9d" event={"ID":"583cecc3-8c60-4c1d-9c94-8a34a4c31d16","Type":"ContainerDied","Data":"e153860a37e21da19b861703a936ba5c76a2fc4acfc2c7f79d893e6b20a77f5f"} Jul 08 23:10:10.766130 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:10.765781 2548 scope.go:117] "RemoveContainer" containerID="c4fe212a96fdf7ddcece7b3ce8d9299b2f3ac21cc613bcf92c26236da6ef5749" Jul 08 23:10:10.774359 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:10.774344 2548 scope.go:117] "RemoveContainer" containerID="c4fe212a96fdf7ddcece7b3ce8d9299b2f3ac21cc613bcf92c26236da6ef5749" Jul 08 23:10:10.774642 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:10:10.774620 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4fe212a96fdf7ddcece7b3ce8d9299b2f3ac21cc613bcf92c26236da6ef5749\": container with ID starting with c4fe212a96fdf7ddcece7b3ce8d9299b2f3ac21cc613bcf92c26236da6ef5749 not found: ID does not exist" containerID="c4fe212a96fdf7ddcece7b3ce8d9299b2f3ac21cc613bcf92c26236da6ef5749" Jul 08 23:10:10.774695 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:10.774654 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4fe212a96fdf7ddcece7b3ce8d9299b2f3ac21cc613bcf92c26236da6ef5749"} err="failed to get container status \"c4fe212a96fdf7ddcece7b3ce8d9299b2f3ac21cc613bcf92c26236da6ef5749\": rpc error: code = NotFound desc = could not find container \"c4fe212a96fdf7ddcece7b3ce8d9299b2f3ac21cc613bcf92c26236da6ef5749\": container with ID starting with c4fe212a96fdf7ddcece7b3ce8d9299b2f3ac21cc613bcf92c26236da6ef5749 not found: ID does not exist" Jul 08 23:10:10.787346 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:10.787315 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-v7z9d"] Jul 08 23:10:10.790677 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:10.790655 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-v7z9d"] Jul 08 23:10:10.828524 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:10.828500 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="583cecc3-8c60-4c1d-9c94-8a34a4c31d16" path="/var/lib/kubelet/pods/583cecc3-8c60-4c1d-9c94-8a34a4c31d16/volumes" Jul 08 23:10:17.271754 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:17.271714 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-r4xs4" Jul 08 23:10:17.272260 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:17.271795 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-r4xs4" Jul 08 23:10:17.294614 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:17.294574 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-r4xs4" Jul 08 23:10:17.818202 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:17.818175 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-r4xs4" Jul 08 23:10:21.989722 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:21.989646 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv"] Jul 08 23:10:21.990067 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:21.989947 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="583cecc3-8c60-4c1d-9c94-8a34a4c31d16" containerName="registry-server" Jul 08 23:10:21.990067 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:21.989959 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="583cecc3-8c60-4c1d-9c94-8a34a4c31d16" containerName="registry-server" Jul 08 23:10:21.990067 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:21.990013 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="583cecc3-8c60-4c1d-9c94-8a34a4c31d16" containerName="registry-server" Jul 08 23:10:21.993118 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:21.993102 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv" Jul 08 23:10:21.995410 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:21.995393 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lh2pm\"" Jul 08 23:10:22.000676 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.000653 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv"] Jul 08 23:10:22.106972 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.106941 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32db00ac-9c9c-40a5-98bd-d995725ab58b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv\" (UID: \"32db00ac-9c9c-40a5-98bd-d995725ab58b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv" Jul 08 23:10:22.107159 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.106981 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h5mq\" (UniqueName: \"kubernetes.io/projected/32db00ac-9c9c-40a5-98bd-d995725ab58b-kube-api-access-4h5mq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv\" (UID: \"32db00ac-9c9c-40a5-98bd-d995725ab58b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv" Jul 08 23:10:22.107159 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.107017 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32db00ac-9c9c-40a5-98bd-d995725ab58b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv\" (UID: \"32db00ac-9c9c-40a5-98bd-d995725ab58b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv" Jul 08 23:10:22.208443 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.208413 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32db00ac-9c9c-40a5-98bd-d995725ab58b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv\" (UID: \"32db00ac-9c9c-40a5-98bd-d995725ab58b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv" Jul 08 23:10:22.208678 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.208521 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32db00ac-9c9c-40a5-98bd-d995725ab58b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv\" (UID: \"32db00ac-9c9c-40a5-98bd-d995725ab58b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv" Jul 08 23:10:22.208678 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.208553 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h5mq\" (UniqueName: \"kubernetes.io/projected/32db00ac-9c9c-40a5-98bd-d995725ab58b-kube-api-access-4h5mq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv\" (UID: \"32db00ac-9c9c-40a5-98bd-d995725ab58b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv" Jul 08 23:10:22.208841 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.208818 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32db00ac-9c9c-40a5-98bd-d995725ab58b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv\" (UID: \"32db00ac-9c9c-40a5-98bd-d995725ab58b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv" Jul 08 23:10:22.208898 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.208864 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32db00ac-9c9c-40a5-98bd-d995725ab58b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv\" (UID: \"32db00ac-9c9c-40a5-98bd-d995725ab58b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv" Jul 08 23:10:22.217093 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.217072 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h5mq\" (UniqueName: \"kubernetes.io/projected/32db00ac-9c9c-40a5-98bd-d995725ab58b-kube-api-access-4h5mq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv\" (UID: \"32db00ac-9c9c-40a5-98bd-d995725ab58b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv" Jul 08 23:10:22.302857 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.302779 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv" Jul 08 23:10:22.431591 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.431545 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv"] Jul 08 23:10:22.433981 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:10:22.433955 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32db00ac_9c9c_40a5_98bd_d995725ab58b.slice/crio-7cb331e75bd3b4c43295292e37f99b7099e9a4a6b99711519778e82d5a5ea581 WatchSource:0}: Error finding container 7cb331e75bd3b4c43295292e37f99b7099e9a4a6b99711519778e82d5a5ea581: Status 404 returned error can't find the container with id 7cb331e75bd3b4c43295292e37f99b7099e9a4a6b99711519778e82d5a5ea581 Jul 08 23:10:22.585559 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.585482 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5"] Jul 08 23:10:22.588775 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.588760 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5" Jul 08 23:10:22.596144 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.596118 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5"] Jul 08 23:10:22.714245 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.714208 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d797d5d-08e0-4ffd-9853-941bb75524cb-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5\" (UID: \"2d797d5d-08e0-4ffd-9853-941bb75524cb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5" Jul 08 23:10:22.714414 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.714268 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx5lq\" (UniqueName: \"kubernetes.io/projected/2d797d5d-08e0-4ffd-9853-941bb75524cb-kube-api-access-mx5lq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5\" (UID: \"2d797d5d-08e0-4ffd-9853-941bb75524cb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5" Jul 08 23:10:22.714414 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.714371 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d797d5d-08e0-4ffd-9853-941bb75524cb-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5\" (UID: \"2d797d5d-08e0-4ffd-9853-941bb75524cb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5" Jul 08 23:10:22.814800 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.814761 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d797d5d-08e0-4ffd-9853-941bb75524cb-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5\" (UID: \"2d797d5d-08e0-4ffd-9853-941bb75524cb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5" Jul 08 23:10:22.814978 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.814814 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d797d5d-08e0-4ffd-9853-941bb75524cb-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5\" (UID: \"2d797d5d-08e0-4ffd-9853-941bb75524cb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5" Jul 08 23:10:22.814978 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.814851 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx5lq\" (UniqueName: \"kubernetes.io/projected/2d797d5d-08e0-4ffd-9853-941bb75524cb-kube-api-access-mx5lq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5\" (UID: \"2d797d5d-08e0-4ffd-9853-941bb75524cb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5" Jul 08 23:10:22.815226 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.815203 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d797d5d-08e0-4ffd-9853-941bb75524cb-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5\" (UID: \"2d797d5d-08e0-4ffd-9853-941bb75524cb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5" Jul 08 23:10:22.815289 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.815233 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d797d5d-08e0-4ffd-9853-941bb75524cb-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5\" (UID: \"2d797d5d-08e0-4ffd-9853-941bb75524cb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5" Jul 08 23:10:22.818619 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.818571 2548 generic.go:356] "Generic (PLEG): container finished" podID="32db00ac-9c9c-40a5-98bd-d995725ab58b" containerID="9ba711024a5ea0c867376cfc5e33498e5989354e311c5bcf89f58419b406f5de" exitCode=0 Jul 08 23:10:22.818714 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.818634 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv" event={"ID":"32db00ac-9c9c-40a5-98bd-d995725ab58b","Type":"ContainerDied","Data":"9ba711024a5ea0c867376cfc5e33498e5989354e311c5bcf89f58419b406f5de"} Jul 08 23:10:22.818772 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.818729 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv" event={"ID":"32db00ac-9c9c-40a5-98bd-d995725ab58b","Type":"ContainerStarted","Data":"7cb331e75bd3b4c43295292e37f99b7099e9a4a6b99711519778e82d5a5ea581"} Jul 08 23:10:22.822464 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.822445 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx5lq\" (UniqueName: \"kubernetes.io/projected/2d797d5d-08e0-4ffd-9853-941bb75524cb-kube-api-access-mx5lq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5\" (UID: \"2d797d5d-08e0-4ffd-9853-941bb75524cb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5" Jul 08 23:10:22.898552 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:22.898460 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5" Jul 08 23:10:23.023512 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.023485 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5"] Jul 08 23:10:23.025497 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:10:23.025473 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d797d5d_08e0_4ffd_9853_941bb75524cb.slice/crio-fe563a3e3275754ea503bf547c7ea11c329100d773a92181446a82df79a91572 WatchSource:0}: Error finding container fe563a3e3275754ea503bf547c7ea11c329100d773a92181446a82df79a91572: Status 404 returned error can't find the container with id fe563a3e3275754ea503bf547c7ea11c329100d773a92181446a82df79a91572 Jul 08 23:10:23.188588 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.188517 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d"] Jul 08 23:10:23.191892 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.191876 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d" Jul 08 23:10:23.199253 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.199224 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d"] Jul 08 23:10:23.319483 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.319443 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df566e6b-9d68-4398-a353-2c510cf87f6d-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d\" (UID: \"df566e6b-9d68-4398-a353-2c510cf87f6d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d" Jul 08 23:10:23.319483 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.319484 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df566e6b-9d68-4398-a353-2c510cf87f6d-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d\" (UID: \"df566e6b-9d68-4398-a353-2c510cf87f6d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d" Jul 08 23:10:23.319721 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.319508 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sgfr\" (UniqueName: \"kubernetes.io/projected/df566e6b-9d68-4398-a353-2c510cf87f6d-kube-api-access-4sgfr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d\" (UID: \"df566e6b-9d68-4398-a353-2c510cf87f6d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d" Jul 08 23:10:23.420876 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.420840 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df566e6b-9d68-4398-a353-2c510cf87f6d-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d\" (UID: \"df566e6b-9d68-4398-a353-2c510cf87f6d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d" Jul 08 23:10:23.421062 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.420891 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df566e6b-9d68-4398-a353-2c510cf87f6d-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d\" (UID: \"df566e6b-9d68-4398-a353-2c510cf87f6d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d" Jul 08 23:10:23.421062 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.420934 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sgfr\" (UniqueName: \"kubernetes.io/projected/df566e6b-9d68-4398-a353-2c510cf87f6d-kube-api-access-4sgfr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d\" (UID: \"df566e6b-9d68-4398-a353-2c510cf87f6d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d" Jul 08 23:10:23.421310 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.421285 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df566e6b-9d68-4398-a353-2c510cf87f6d-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d\" (UID: \"df566e6b-9d68-4398-a353-2c510cf87f6d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d" Jul 08 23:10:23.421387 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.421306 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df566e6b-9d68-4398-a353-2c510cf87f6d-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d\" (UID: \"df566e6b-9d68-4398-a353-2c510cf87f6d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d" Jul 08 23:10:23.428691 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.428664 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sgfr\" (UniqueName: \"kubernetes.io/projected/df566e6b-9d68-4398-a353-2c510cf87f6d-kube-api-access-4sgfr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d\" (UID: \"df566e6b-9d68-4398-a353-2c510cf87f6d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d" Jul 08 23:10:23.501713 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.501682 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d" Jul 08 23:10:23.592756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.592728 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7"] Jul 08 23:10:23.597689 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.597667 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7" Jul 08 23:10:23.604242 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.604219 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7"] Jul 08 23:10:23.641421 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.641393 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d"] Jul 08 23:10:23.645023 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:10:23.645001 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf566e6b_9d68_4398_a353_2c510cf87f6d.slice/crio-652aac16ecdb46ab513b49e2eb6d9e46a03ba612c931a66761694d34e53f6a64 WatchSource:0}: Error finding container 652aac16ecdb46ab513b49e2eb6d9e46a03ba612c931a66761694d34e53f6a64: Status 404 returned error can't find the container with id 652aac16ecdb46ab513b49e2eb6d9e46a03ba612c931a66761694d34e53f6a64 Jul 08 23:10:23.722481 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.722452 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff5hl\" (UniqueName: \"kubernetes.io/projected/85157b14-97af-430b-9a83-295bfd02218e-kube-api-access-ff5hl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7\" (UID: \"85157b14-97af-430b-9a83-295bfd02218e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7" Jul 08 23:10:23.722581 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.722499 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85157b14-97af-430b-9a83-295bfd02218e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7\" (UID: \"85157b14-97af-430b-9a83-295bfd02218e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7" Jul 08 23:10:23.722645 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.722585 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85157b14-97af-430b-9a83-295bfd02218e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7\" (UID: \"85157b14-97af-430b-9a83-295bfd02218e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7" Jul 08 23:10:23.823800 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.823769 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff5hl\" (UniqueName: \"kubernetes.io/projected/85157b14-97af-430b-9a83-295bfd02218e-kube-api-access-ff5hl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7\" (UID: \"85157b14-97af-430b-9a83-295bfd02218e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7" Jul 08 23:10:23.823972 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.823837 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85157b14-97af-430b-9a83-295bfd02218e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7\" (UID: \"85157b14-97af-430b-9a83-295bfd02218e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7" Jul 08 23:10:23.823972 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.823883 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85157b14-97af-430b-9a83-295bfd02218e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7\" (UID: \"85157b14-97af-430b-9a83-295bfd02218e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7" Jul 08 23:10:23.824248 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.824217 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85157b14-97af-430b-9a83-295bfd02218e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7\" (UID: \"85157b14-97af-430b-9a83-295bfd02218e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7" Jul 08 23:10:23.824379 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.824244 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85157b14-97af-430b-9a83-295bfd02218e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7\" (UID: \"85157b14-97af-430b-9a83-295bfd02218e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7" Jul 08 23:10:23.825098 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.825073 2548 generic.go:356] "Generic (PLEG): container finished" podID="df566e6b-9d68-4398-a353-2c510cf87f6d" containerID="9d37996373ebf5ffe5f0f2587150bf5bddd241af8c90bbd88a9d8ad514053de1" exitCode=0 Jul 08 23:10:23.825197 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.825159 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d" event={"ID":"df566e6b-9d68-4398-a353-2c510cf87f6d","Type":"ContainerDied","Data":"9d37996373ebf5ffe5f0f2587150bf5bddd241af8c90bbd88a9d8ad514053de1"} Jul 08 23:10:23.825253 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.825203 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d" event={"ID":"df566e6b-9d68-4398-a353-2c510cf87f6d","Type":"ContainerStarted","Data":"652aac16ecdb46ab513b49e2eb6d9e46a03ba612c931a66761694d34e53f6a64"} Jul 08 23:10:23.827098 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.827071 2548 generic.go:356] "Generic (PLEG): container finished" podID="32db00ac-9c9c-40a5-98bd-d995725ab58b" containerID="a6238631bcefa45cfa8284f40935cbaaaf6735811f7ecbdfbdfea3fe41590ac1" exitCode=0 Jul 08 23:10:23.827199 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.827155 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv" event={"ID":"32db00ac-9c9c-40a5-98bd-d995725ab58b","Type":"ContainerDied","Data":"a6238631bcefa45cfa8284f40935cbaaaf6735811f7ecbdfbdfea3fe41590ac1"} Jul 08 23:10:23.828802 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.828782 2548 generic.go:356] "Generic (PLEG): container finished" podID="2d797d5d-08e0-4ffd-9853-941bb75524cb" containerID="9bf566afac0a640cbaa42224e55ae6f61b621b9acb1b5867abe750c7c55e7b38" exitCode=0 Jul 08 23:10:23.828880 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.828854 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5" event={"ID":"2d797d5d-08e0-4ffd-9853-941bb75524cb","Type":"ContainerDied","Data":"9bf566afac0a640cbaa42224e55ae6f61b621b9acb1b5867abe750c7c55e7b38"} Jul 08 23:10:23.828924 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.828892 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5" event={"ID":"2d797d5d-08e0-4ffd-9853-941bb75524cb","Type":"ContainerStarted","Data":"fe563a3e3275754ea503bf547c7ea11c329100d773a92181446a82df79a91572"} Jul 08 23:10:23.832397 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.832378 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff5hl\" (UniqueName: \"kubernetes.io/projected/85157b14-97af-430b-9a83-295bfd02218e-kube-api-access-ff5hl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7\" (UID: \"85157b14-97af-430b-9a83-295bfd02218e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7" Jul 08 23:10:23.911140 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:23.911109 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7" Jul 08 23:10:24.043580 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:24.043554 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7"] Jul 08 23:10:24.045494 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:10:24.045466 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85157b14_97af_430b_9a83_295bfd02218e.slice/crio-d56cfe51bf561fc060d2b6c269002672af0691119c2904d6b034be660207fa03 WatchSource:0}: Error finding container d56cfe51bf561fc060d2b6c269002672af0691119c2904d6b034be660207fa03: Status 404 returned error can't find the container with id d56cfe51bf561fc060d2b6c269002672af0691119c2904d6b034be660207fa03 Jul 08 23:10:24.838251 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:24.838226 2548 generic.go:356] "Generic (PLEG): container finished" podID="32db00ac-9c9c-40a5-98bd-d995725ab58b" containerID="1f90f6783a7c42c254d4291dda6c4682ef288ef7b51b080c125f9acf1371c89e" exitCode=0 Jul 08 23:10:24.838360 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:24.838308 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv" event={"ID":"32db00ac-9c9c-40a5-98bd-d995725ab58b","Type":"ContainerDied","Data":"1f90f6783a7c42c254d4291dda6c4682ef288ef7b51b080c125f9acf1371c89e"} Jul 08 23:10:24.839779 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:24.839757 2548 generic.go:356] "Generic (PLEG): container finished" podID="85157b14-97af-430b-9a83-295bfd02218e" containerID="942574aa6ca0044c6d221de731c2fe79faf8aad0aab032daaa7265b710fd7dbc" exitCode=0 Jul 08 23:10:24.839875 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:24.839840 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7" event={"ID":"85157b14-97af-430b-9a83-295bfd02218e","Type":"ContainerDied","Data":"942574aa6ca0044c6d221de731c2fe79faf8aad0aab032daaa7265b710fd7dbc"} Jul 08 23:10:24.839938 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:24.839882 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7" event={"ID":"85157b14-97af-430b-9a83-295bfd02218e","Type":"ContainerStarted","Data":"d56cfe51bf561fc060d2b6c269002672af0691119c2904d6b034be660207fa03"} Jul 08 23:10:25.846406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:25.846373 2548 generic.go:356] "Generic (PLEG): container finished" podID="2d797d5d-08e0-4ffd-9853-941bb75524cb" containerID="36442d58e602990e2029b719dc46e9f32f5347fe29d3c44fc203ebd54cf62453" exitCode=0 Jul 08 23:10:25.846876 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:25.846429 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5" event={"ID":"2d797d5d-08e0-4ffd-9853-941bb75524cb","Type":"ContainerDied","Data":"36442d58e602990e2029b719dc46e9f32f5347fe29d3c44fc203ebd54cf62453"} Jul 08 23:10:25.848394 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:25.848372 2548 generic.go:356] "Generic (PLEG): container finished" podID="85157b14-97af-430b-9a83-295bfd02218e" containerID="6dbbdfd71315fd12dff0223ce658e65735d4c327c7c2b04ca4cfe0814293f9a3" exitCode=0 Jul 08 23:10:25.848518 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:25.848496 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7" event={"ID":"85157b14-97af-430b-9a83-295bfd02218e","Type":"ContainerDied","Data":"6dbbdfd71315fd12dff0223ce658e65735d4c327c7c2b04ca4cfe0814293f9a3"} Jul 08 23:10:25.850440 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:25.850418 2548 generic.go:356] "Generic (PLEG): container finished" podID="df566e6b-9d68-4398-a353-2c510cf87f6d" containerID="49f83d95f265c32da2fb00f717c889f87b643a6d0621f1880e390ee8e5cd995a" exitCode=0 Jul 08 23:10:25.850523 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:25.850448 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d" event={"ID":"df566e6b-9d68-4398-a353-2c510cf87f6d","Type":"ContainerDied","Data":"49f83d95f265c32da2fb00f717c889f87b643a6d0621f1880e390ee8e5cd995a"} Jul 08 23:10:25.991425 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:25.991399 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv" Jul 08 23:10:26.146982 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:26.146948 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32db00ac-9c9c-40a5-98bd-d995725ab58b-bundle\") pod \"32db00ac-9c9c-40a5-98bd-d995725ab58b\" (UID: \"32db00ac-9c9c-40a5-98bd-d995725ab58b\") " Jul 08 23:10:26.147134 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:26.147003 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4h5mq\" (UniqueName: \"kubernetes.io/projected/32db00ac-9c9c-40a5-98bd-d995725ab58b-kube-api-access-4h5mq\") pod \"32db00ac-9c9c-40a5-98bd-d995725ab58b\" (UID: \"32db00ac-9c9c-40a5-98bd-d995725ab58b\") " Jul 08 23:10:26.147134 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:26.147033 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32db00ac-9c9c-40a5-98bd-d995725ab58b-util\") pod \"32db00ac-9c9c-40a5-98bd-d995725ab58b\" (UID: \"32db00ac-9c9c-40a5-98bd-d995725ab58b\") " Jul 08 23:10:26.147588 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:26.147550 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32db00ac-9c9c-40a5-98bd-d995725ab58b-bundle" (OuterVolumeSpecName: "bundle") pod "32db00ac-9c9c-40a5-98bd-d995725ab58b" (UID: "32db00ac-9c9c-40a5-98bd-d995725ab58b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:10:26.149041 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:26.149019 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32db00ac-9c9c-40a5-98bd-d995725ab58b-kube-api-access-4h5mq" (OuterVolumeSpecName: "kube-api-access-4h5mq") pod "32db00ac-9c9c-40a5-98bd-d995725ab58b" (UID: "32db00ac-9c9c-40a5-98bd-d995725ab58b"). InnerVolumeSpecName "kube-api-access-4h5mq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:10:26.153057 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:26.153032 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32db00ac-9c9c-40a5-98bd-d995725ab58b-util" (OuterVolumeSpecName: "util") pod "32db00ac-9c9c-40a5-98bd-d995725ab58b" (UID: "32db00ac-9c9c-40a5-98bd-d995725ab58b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:10:26.248416 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:26.248383 2548 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32db00ac-9c9c-40a5-98bd-d995725ab58b-util\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:10:26.248416 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:26.248413 2548 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32db00ac-9c9c-40a5-98bd-d995725ab58b-bundle\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:10:26.248571 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:26.248422 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4h5mq\" (UniqueName: \"kubernetes.io/projected/32db00ac-9c9c-40a5-98bd-d995725ab58b-kube-api-access-4h5mq\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:10:26.856750 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:26.856727 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv" Jul 08 23:10:26.857136 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:26.856723 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv" event={"ID":"32db00ac-9c9c-40a5-98bd-d995725ab58b","Type":"ContainerDied","Data":"7cb331e75bd3b4c43295292e37f99b7099e9a4a6b99711519778e82d5a5ea581"} Jul 08 23:10:26.857136 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:26.856858 2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7cb331e75bd3b4c43295292e37f99b7099e9a4a6b99711519778e82d5a5ea581" Jul 08 23:10:26.858954 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:26.858927 2548 generic.go:356] "Generic (PLEG): container finished" podID="2d797d5d-08e0-4ffd-9853-941bb75524cb" containerID="598c3a8f4d74e233ebcbe12ddb7d4563391f08d0c7c7d6498f354748f4b8791f" exitCode=0 Jul 08 23:10:26.859079 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:26.859007 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5" event={"ID":"2d797d5d-08e0-4ffd-9853-941bb75524cb","Type":"ContainerDied","Data":"598c3a8f4d74e233ebcbe12ddb7d4563391f08d0c7c7d6498f354748f4b8791f"} Jul 08 23:10:26.860976 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:26.860952 2548 generic.go:356] "Generic (PLEG): container finished" podID="85157b14-97af-430b-9a83-295bfd02218e" containerID="990b691eec1d56fe0197459acdba3b207c8378a11cca7e2a8f7d6c09d230fb88" exitCode=0 Jul 08 23:10:26.861071 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:26.861025 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7" event={"ID":"85157b14-97af-430b-9a83-295bfd02218e","Type":"ContainerDied","Data":"990b691eec1d56fe0197459acdba3b207c8378a11cca7e2a8f7d6c09d230fb88"} Jul 08 23:10:26.863073 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:26.863054 2548 generic.go:356] "Generic (PLEG): container finished" podID="df566e6b-9d68-4398-a353-2c510cf87f6d" containerID="56c809ed8fde7dd5dc54adfc61adcae2228ca01465a744514cfe1ff2c25a4a1e" exitCode=0 Jul 08 23:10:26.863159 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:26.863096 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d" event={"ID":"df566e6b-9d68-4398-a353-2c510cf87f6d","Type":"ContainerDied","Data":"56c809ed8fde7dd5dc54adfc61adcae2228ca01465a744514cfe1ff2c25a4a1e"} Jul 08 23:10:28.018333 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.018305 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d" Jul 08 23:10:28.087961 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.087939 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5" Jul 08 23:10:28.090698 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.090678 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7" Jul 08 23:10:28.167464 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.167386 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mx5lq\" (UniqueName: \"kubernetes.io/projected/2d797d5d-08e0-4ffd-9853-941bb75524cb-kube-api-access-mx5lq\") pod \"2d797d5d-08e0-4ffd-9853-941bb75524cb\" (UID: \"2d797d5d-08e0-4ffd-9853-941bb75524cb\") " Jul 08 23:10:28.167464 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.167442 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d797d5d-08e0-4ffd-9853-941bb75524cb-bundle\") pod \"2d797d5d-08e0-4ffd-9853-941bb75524cb\" (UID: \"2d797d5d-08e0-4ffd-9853-941bb75524cb\") " Jul 08 23:10:28.167722 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.167469 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85157b14-97af-430b-9a83-295bfd02218e-bundle\") pod \"85157b14-97af-430b-9a83-295bfd02218e\" (UID: \"85157b14-97af-430b-9a83-295bfd02218e\") " Jul 08 23:10:28.167722 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.167489 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d797d5d-08e0-4ffd-9853-941bb75524cb-util\") pod \"2d797d5d-08e0-4ffd-9853-941bb75524cb\" (UID: \"2d797d5d-08e0-4ffd-9853-941bb75524cb\") " Jul 08 23:10:28.167722 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.167505 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85157b14-97af-430b-9a83-295bfd02218e-util\") pod \"85157b14-97af-430b-9a83-295bfd02218e\" (UID: \"85157b14-97af-430b-9a83-295bfd02218e\") " Jul 08 23:10:28.167722 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.167555 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df566e6b-9d68-4398-a353-2c510cf87f6d-bundle\") pod \"df566e6b-9d68-4398-a353-2c510cf87f6d\" (UID: \"df566e6b-9d68-4398-a353-2c510cf87f6d\") " Jul 08 23:10:28.167722 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.167580 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4sgfr\" (UniqueName: \"kubernetes.io/projected/df566e6b-9d68-4398-a353-2c510cf87f6d-kube-api-access-4sgfr\") pod \"df566e6b-9d68-4398-a353-2c510cf87f6d\" (UID: \"df566e6b-9d68-4398-a353-2c510cf87f6d\") " Jul 08 23:10:28.167722 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.167636 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df566e6b-9d68-4398-a353-2c510cf87f6d-util\") pod \"df566e6b-9d68-4398-a353-2c510cf87f6d\" (UID: \"df566e6b-9d68-4398-a353-2c510cf87f6d\") " Jul 08 23:10:28.167722 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.167689 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ff5hl\" (UniqueName: \"kubernetes.io/projected/85157b14-97af-430b-9a83-295bfd02218e-kube-api-access-ff5hl\") pod \"85157b14-97af-430b-9a83-295bfd02218e\" (UID: \"85157b14-97af-430b-9a83-295bfd02218e\") " Jul 08 23:10:28.168283 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.168226 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d797d5d-08e0-4ffd-9853-941bb75524cb-bundle" (OuterVolumeSpecName: "bundle") pod "2d797d5d-08e0-4ffd-9853-941bb75524cb" (UID: "2d797d5d-08e0-4ffd-9853-941bb75524cb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:10:28.168397 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.168355 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85157b14-97af-430b-9a83-295bfd02218e-bundle" (OuterVolumeSpecName: "bundle") pod "85157b14-97af-430b-9a83-295bfd02218e" (UID: "85157b14-97af-430b-9a83-295bfd02218e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:10:28.168511 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.168481 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df566e6b-9d68-4398-a353-2c510cf87f6d-bundle" (OuterVolumeSpecName: "bundle") pod "df566e6b-9d68-4398-a353-2c510cf87f6d" (UID: "df566e6b-9d68-4398-a353-2c510cf87f6d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:10:28.170020 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.169976 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d797d5d-08e0-4ffd-9853-941bb75524cb-kube-api-access-mx5lq" (OuterVolumeSpecName: "kube-api-access-mx5lq") pod "2d797d5d-08e0-4ffd-9853-941bb75524cb" (UID: "2d797d5d-08e0-4ffd-9853-941bb75524cb"). InnerVolumeSpecName "kube-api-access-mx5lq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:10:28.170365 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.170333 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85157b14-97af-430b-9a83-295bfd02218e-kube-api-access-ff5hl" (OuterVolumeSpecName: "kube-api-access-ff5hl") pod "85157b14-97af-430b-9a83-295bfd02218e" (UID: "85157b14-97af-430b-9a83-295bfd02218e"). InnerVolumeSpecName "kube-api-access-ff5hl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:10:28.170466 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.170431 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df566e6b-9d68-4398-a353-2c510cf87f6d-kube-api-access-4sgfr" (OuterVolumeSpecName: "kube-api-access-4sgfr") pod "df566e6b-9d68-4398-a353-2c510cf87f6d" (UID: "df566e6b-9d68-4398-a353-2c510cf87f6d"). InnerVolumeSpecName "kube-api-access-4sgfr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:10:28.172990 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.172971 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85157b14-97af-430b-9a83-295bfd02218e-util" (OuterVolumeSpecName: "util") pod "85157b14-97af-430b-9a83-295bfd02218e" (UID: "85157b14-97af-430b-9a83-295bfd02218e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:10:28.174019 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.173999 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d797d5d-08e0-4ffd-9853-941bb75524cb-util" (OuterVolumeSpecName: "util") pod "2d797d5d-08e0-4ffd-9853-941bb75524cb" (UID: "2d797d5d-08e0-4ffd-9853-941bb75524cb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:10:28.175121 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.175092 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df566e6b-9d68-4398-a353-2c510cf87f6d-util" (OuterVolumeSpecName: "util") pod "df566e6b-9d68-4398-a353-2c510cf87f6d" (UID: "df566e6b-9d68-4398-a353-2c510cf87f6d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:10:28.268484 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.268451 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mx5lq\" (UniqueName: \"kubernetes.io/projected/2d797d5d-08e0-4ffd-9853-941bb75524cb-kube-api-access-mx5lq\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:10:28.268484 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.268480 2548 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d797d5d-08e0-4ffd-9853-941bb75524cb-bundle\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:10:28.268484 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.268491 2548 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85157b14-97af-430b-9a83-295bfd02218e-bundle\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:10:28.268736 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.268498 2548 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d797d5d-08e0-4ffd-9853-941bb75524cb-util\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:10:28.268736 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.268506 2548 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85157b14-97af-430b-9a83-295bfd02218e-util\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:10:28.268736 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.268513 2548 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df566e6b-9d68-4398-a353-2c510cf87f6d-bundle\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:10:28.268736 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.268522 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4sgfr\" (UniqueName: \"kubernetes.io/projected/df566e6b-9d68-4398-a353-2c510cf87f6d-kube-api-access-4sgfr\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:10:28.268736 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.268530 2548 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df566e6b-9d68-4398-a353-2c510cf87f6d-util\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:10:28.268736 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.268538 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ff5hl\" (UniqueName: \"kubernetes.io/projected/85157b14-97af-430b-9a83-295bfd02218e-kube-api-access-ff5hl\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:10:28.866250 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:10:28.866220 2548 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf566e6b_9d68_4398_a353_2c510cf87f6d.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d797d5d_08e0_4ffd_9853_941bb75524cb.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85157b14_97af_430b_9a83_295bfd02218e.slice\": RecentStats: unable to find data in memory cache]" Jul 08 23:10:28.876545 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.876524 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d" Jul 08 23:10:28.876545 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.876530 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d" event={"ID":"df566e6b-9d68-4398-a353-2c510cf87f6d","Type":"ContainerDied","Data":"652aac16ecdb46ab513b49e2eb6d9e46a03ba612c931a66761694d34e53f6a64"} Jul 08 23:10:28.876708 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.876572 2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="652aac16ecdb46ab513b49e2eb6d9e46a03ba612c931a66761694d34e53f6a64" Jul 08 23:10:28.878449 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.878430 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5" Jul 08 23:10:28.878548 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.878440 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5" event={"ID":"2d797d5d-08e0-4ffd-9853-941bb75524cb","Type":"ContainerDied","Data":"fe563a3e3275754ea503bf547c7ea11c329100d773a92181446a82df79a91572"} Jul 08 23:10:28.878548 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.878485 2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe563a3e3275754ea503bf547c7ea11c329100d773a92181446a82df79a91572" Jul 08 23:10:28.880456 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.880404 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7" event={"ID":"85157b14-97af-430b-9a83-295bfd02218e","Type":"ContainerDied","Data":"d56cfe51bf561fc060d2b6c269002672af0691119c2904d6b034be660207fa03"} Jul 08 23:10:28.880456 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.880447 2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d56cfe51bf561fc060d2b6c269002672af0691119c2904d6b034be660207fa03" Jul 08 23:10:28.880589 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:28.880458 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7" Jul 08 23:10:38.995267 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995231 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tchdl"] Jul 08 23:10:38.995697 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995568 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85157b14-97af-430b-9a83-295bfd02218e" containerName="util" Jul 08 23:10:38.995697 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995582 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="85157b14-97af-430b-9a83-295bfd02218e" containerName="util" Jul 08 23:10:38.995697 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995607 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d797d5d-08e0-4ffd-9853-941bb75524cb" containerName="extract" Jul 08 23:10:38.995697 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995613 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d797d5d-08e0-4ffd-9853-941bb75524cb" containerName="extract" Jul 08 23:10:38.995697 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995626 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32db00ac-9c9c-40a5-98bd-d995725ab58b" containerName="pull" Jul 08 23:10:38.995697 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995631 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="32db00ac-9c9c-40a5-98bd-d995725ab58b" containerName="pull" Jul 08 23:10:38.995697 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995636 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df566e6b-9d68-4398-a353-2c510cf87f6d" containerName="pull" Jul 08 23:10:38.995697 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995641 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="df566e6b-9d68-4398-a353-2c510cf87f6d" containerName="pull" Jul 08 23:10:38.995697 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995647 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85157b14-97af-430b-9a83-295bfd02218e" containerName="extract" Jul 08 23:10:38.995697 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995653 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="85157b14-97af-430b-9a83-295bfd02218e" containerName="extract" Jul 08 23:10:38.995697 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995677 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df566e6b-9d68-4398-a353-2c510cf87f6d" containerName="extract" Jul 08 23:10:38.995697 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995682 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="df566e6b-9d68-4398-a353-2c510cf87f6d" containerName="extract" Jul 08 23:10:38.995697 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995688 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32db00ac-9c9c-40a5-98bd-d995725ab58b" containerName="util" Jul 08 23:10:38.995697 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995692 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="32db00ac-9c9c-40a5-98bd-d995725ab58b" containerName="util" Jul 08 23:10:38.995697 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995697 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32db00ac-9c9c-40a5-98bd-d995725ab58b" containerName="extract" Jul 08 23:10:38.995697 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995702 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="32db00ac-9c9c-40a5-98bd-d995725ab58b" containerName="extract" Jul 08 23:10:38.996182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995709 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d797d5d-08e0-4ffd-9853-941bb75524cb" containerName="util" Jul 08 23:10:38.996182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995714 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d797d5d-08e0-4ffd-9853-941bb75524cb" containerName="util" Jul 08 23:10:38.996182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995721 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df566e6b-9d68-4398-a353-2c510cf87f6d" containerName="util" Jul 08 23:10:38.996182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995726 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="df566e6b-9d68-4398-a353-2c510cf87f6d" containerName="util" Jul 08 23:10:38.996182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995731 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85157b14-97af-430b-9a83-295bfd02218e" containerName="pull" Jul 08 23:10:38.996182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995735 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="85157b14-97af-430b-9a83-295bfd02218e" containerName="pull" Jul 08 23:10:38.996182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995744 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d797d5d-08e0-4ffd-9853-941bb75524cb" containerName="pull" Jul 08 23:10:38.996182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995752 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d797d5d-08e0-4ffd-9853-941bb75524cb" containerName="pull" Jul 08 23:10:38.996182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995801 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d797d5d-08e0-4ffd-9853-941bb75524cb" containerName="extract" Jul 08 23:10:38.996182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995809 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="df566e6b-9d68-4398-a353-2c510cf87f6d" containerName="extract" Jul 08 23:10:38.996182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995815 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="32db00ac-9c9c-40a5-98bd-d995725ab58b" containerName="extract" Jul 08 23:10:38.996182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:38.995822 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="85157b14-97af-430b-9a83-295bfd02218e" containerName="extract" Jul 08 23:10:39.000251 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:39.000231 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tchdl" Jul 08 23:10:39.002781 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:39.002759 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-fz95t\"" Jul 08 23:10:39.007804 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:39.007779 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tchdl"] Jul 08 23:10:39.161238 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:39.161207 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4zbx\" (UniqueName: \"kubernetes.io/projected/ad3c68fb-f770-4f70-b7b4-8016f5262f74-kube-api-access-c4zbx\") pod \"authorino-operator-75cc8d5cb7-tchdl\" (UID: \"ad3c68fb-f770-4f70-b7b4-8016f5262f74\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tchdl" Jul 08 23:10:39.262522 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:39.262430 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4zbx\" (UniqueName: \"kubernetes.io/projected/ad3c68fb-f770-4f70-b7b4-8016f5262f74-kube-api-access-c4zbx\") pod \"authorino-operator-75cc8d5cb7-tchdl\" (UID: \"ad3c68fb-f770-4f70-b7b4-8016f5262f74\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tchdl" Jul 08 23:10:39.270514 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:39.270491 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4zbx\" (UniqueName: \"kubernetes.io/projected/ad3c68fb-f770-4f70-b7b4-8016f5262f74-kube-api-access-c4zbx\") pod \"authorino-operator-75cc8d5cb7-tchdl\" (UID: \"ad3c68fb-f770-4f70-b7b4-8016f5262f74\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tchdl" Jul 08 23:10:39.311517 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:39.311463 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tchdl" Jul 08 23:10:39.442893 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:39.442869 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tchdl"] Jul 08 23:10:39.445235 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:10:39.445211 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad3c68fb_f770_4f70_b7b4_8016f5262f74.slice/crio-d7be08e7a8cd6ec8ac181ad267d87108ce5f3237aab3b52048745a96c3c7a023 WatchSource:0}: Error finding container d7be08e7a8cd6ec8ac181ad267d87108ce5f3237aab3b52048745a96c3c7a023: Status 404 returned error can't find the container with id d7be08e7a8cd6ec8ac181ad267d87108ce5f3237aab3b52048745a96c3c7a023 Jul 08 23:10:39.934340 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:39.934303 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b4b74587c-ccgb8"] Jul 08 23:10:39.939006 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:39.938980 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tchdl" event={"ID":"ad3c68fb-f770-4f70-b7b4-8016f5262f74","Type":"ContainerStarted","Data":"d7be08e7a8cd6ec8ac181ad267d87108ce5f3237aab3b52048745a96c3c7a023"} Jul 08 23:10:39.939142 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:39.939067 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:39.949859 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:39.949833 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b4b74587c-ccgb8"] Jul 08 23:10:39.967980 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:39.967952 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/052f84cf-54ae-4565-a35a-3d7ed3a53f63-oauth-serving-cert\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:39.968158 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:39.968005 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8phr\" (UniqueName: \"kubernetes.io/projected/052f84cf-54ae-4565-a35a-3d7ed3a53f63-kube-api-access-g8phr\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:39.968158 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:39.968051 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/052f84cf-54ae-4565-a35a-3d7ed3a53f63-console-serving-cert\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:39.968158 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:39.968086 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/052f84cf-54ae-4565-a35a-3d7ed3a53f63-trusted-ca-bundle\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:39.968158 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:39.968107 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/052f84cf-54ae-4565-a35a-3d7ed3a53f63-service-ca\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:39.968158 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:39.968130 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/052f84cf-54ae-4565-a35a-3d7ed3a53f63-console-config\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:39.968326 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:39.968229 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/052f84cf-54ae-4565-a35a-3d7ed3a53f63-console-oauth-config\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:40.069436 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:40.069405 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/052f84cf-54ae-4565-a35a-3d7ed3a53f63-console-serving-cert\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:40.069879 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:40.069452 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/052f84cf-54ae-4565-a35a-3d7ed3a53f63-trusted-ca-bundle\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:40.069879 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:40.069492 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/052f84cf-54ae-4565-a35a-3d7ed3a53f63-service-ca\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:40.069879 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:40.069513 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/052f84cf-54ae-4565-a35a-3d7ed3a53f63-console-config\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:40.069879 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:40.069552 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/052f84cf-54ae-4565-a35a-3d7ed3a53f63-console-oauth-config\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:40.069879 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:40.069573 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/052f84cf-54ae-4565-a35a-3d7ed3a53f63-oauth-serving-cert\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:40.069879 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:40.069633 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8phr\" (UniqueName: \"kubernetes.io/projected/052f84cf-54ae-4565-a35a-3d7ed3a53f63-kube-api-access-g8phr\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:40.070349 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:40.070321 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/052f84cf-54ae-4565-a35a-3d7ed3a53f63-service-ca\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:40.070457 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:40.070350 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/052f84cf-54ae-4565-a35a-3d7ed3a53f63-oauth-serving-cert\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:40.070457 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:40.070356 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/052f84cf-54ae-4565-a35a-3d7ed3a53f63-console-config\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:40.070641 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:40.070621 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/052f84cf-54ae-4565-a35a-3d7ed3a53f63-trusted-ca-bundle\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:40.071958 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:40.071936 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/052f84cf-54ae-4565-a35a-3d7ed3a53f63-console-oauth-config\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:40.072066 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:40.072044 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/052f84cf-54ae-4565-a35a-3d7ed3a53f63-console-serving-cert\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:40.079919 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:40.079897 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8phr\" (UniqueName: \"kubernetes.io/projected/052f84cf-54ae-4565-a35a-3d7ed3a53f63-kube-api-access-g8phr\") pod \"console-b4b74587c-ccgb8\" (UID: \"052f84cf-54ae-4565-a35a-3d7ed3a53f63\") " pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:40.249009 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:40.248975 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:40.430703 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:40.430672 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b4b74587c-ccgb8"] Jul 08 23:10:40.433360 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:10:40.433330 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod052f84cf_54ae_4565_a35a_3d7ed3a53f63.slice/crio-e5b2dbe315f5bed763336749c91dd1c98b232deac75cfb73bd88b3e5287b6632 WatchSource:0}: Error finding container e5b2dbe315f5bed763336749c91dd1c98b232deac75cfb73bd88b3e5287b6632: Status 404 returned error can't find the container with id e5b2dbe315f5bed763336749c91dd1c98b232deac75cfb73bd88b3e5287b6632 Jul 08 23:10:40.943009 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:40.942928 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b4b74587c-ccgb8" event={"ID":"052f84cf-54ae-4565-a35a-3d7ed3a53f63","Type":"ContainerStarted","Data":"b38ed39b44b94bbe4206d4c213f24e68b5b6ddd3ed89b6dfcac4d0f74c81c23b"} Jul 08 23:10:40.943009 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:40.942989 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b4b74587c-ccgb8" event={"ID":"052f84cf-54ae-4565-a35a-3d7ed3a53f63","Type":"ContainerStarted","Data":"e5b2dbe315f5bed763336749c91dd1c98b232deac75cfb73bd88b3e5287b6632"} Jul 08 23:10:40.961666 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:40.961611 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b4b74587c-ccgb8" podStartSLOduration=1.961581126 podStartE2EDuration="1.961581126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:10:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:10:40.960153195 +0000 UTC m=+462.720648250" watchObservedRunningTime="2026-07-08 23:10:40.961581126 +0000 UTC m=+462.722076181" Jul 08 23:10:41.949839 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:41.949804 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tchdl" event={"ID":"ad3c68fb-f770-4f70-b7b4-8016f5262f74","Type":"ContainerStarted","Data":"b1d77827d1ade995685da332286773cf941ca98c1d4d666af78a3387dd399c55"} Jul 08 23:10:41.969330 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:41.969253 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tchdl" podStartSLOduration=1.6378054469999999 podStartE2EDuration="3.969232513s" totalImagesPullingTime="2.331427066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:10:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:10:41.966800295 +0000 UTC m=+463.727295360" watchObservedRunningTime="2026-07-08 23:10:41.969232513 +0000 UTC m=+463.729727569" Jul 08 23:10:42.271182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:42.271139 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v6wdk"] Jul 08 23:10:42.280560 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:42.280521 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v6wdk" Jul 08 23:10:42.283114 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:42.283070 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-m8jsz\"" Jul 08 23:10:42.285376 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:42.285349 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v6wdk"] Jul 08 23:10:42.290830 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:42.290809 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncdtr\" (UniqueName: \"kubernetes.io/projected/5cb2a2a0-b852-4d71-b5b6-fa360d2279ef-kube-api-access-ncdtr\") pod \"limitador-operator-controller-manager-859d7b78bd-v6wdk\" (UID: \"5cb2a2a0-b852-4d71-b5b6-fa360d2279ef\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v6wdk" Jul 08 23:10:42.391249 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:42.391219 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncdtr\" (UniqueName: \"kubernetes.io/projected/5cb2a2a0-b852-4d71-b5b6-fa360d2279ef-kube-api-access-ncdtr\") pod \"limitador-operator-controller-manager-859d7b78bd-v6wdk\" (UID: \"5cb2a2a0-b852-4d71-b5b6-fa360d2279ef\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v6wdk" Jul 08 23:10:42.400216 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:42.400184 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncdtr\" (UniqueName: \"kubernetes.io/projected/5cb2a2a0-b852-4d71-b5b6-fa360d2279ef-kube-api-access-ncdtr\") pod \"limitador-operator-controller-manager-859d7b78bd-v6wdk\" (UID: \"5cb2a2a0-b852-4d71-b5b6-fa360d2279ef\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v6wdk" Jul 08 23:10:42.591851 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:42.591766 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v6wdk" Jul 08 23:10:42.734967 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:42.734863 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v6wdk"] Jul 08 23:10:42.739326 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:10:42.739286 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cb2a2a0_b852_4d71_b5b6_fa360d2279ef.slice/crio-2d35dfe3aa06ebe2239286647c6a884c08ff43da7a118e7da08ff780c3cb7949 WatchSource:0}: Error finding container 2d35dfe3aa06ebe2239286647c6a884c08ff43da7a118e7da08ff780c3cb7949: Status 404 returned error can't find the container with id 2d35dfe3aa06ebe2239286647c6a884c08ff43da7a118e7da08ff780c3cb7949 Jul 08 23:10:42.955399 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:42.955311 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v6wdk" event={"ID":"5cb2a2a0-b852-4d71-b5b6-fa360d2279ef","Type":"ContainerStarted","Data":"2d35dfe3aa06ebe2239286647c6a884c08ff43da7a118e7da08ff780c3cb7949"} Jul 08 23:10:42.955809 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:42.955457 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tchdl" Jul 08 23:10:45.972167 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:45.972132 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v6wdk" event={"ID":"5cb2a2a0-b852-4d71-b5b6-fa360d2279ef","Type":"ContainerStarted","Data":"b01aafbde3fc34736a780cc195f94f6db50d10e33141c01be8b0c511e601d597"} Jul 08 23:10:45.972660 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:45.972263 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v6wdk" Jul 08 23:10:45.990769 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:45.990721 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v6wdk" podStartSLOduration=1.8053246779999998 podStartE2EDuration="3.99070581s" totalImagesPullingTime="2.185381132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:10:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:10:45.988478646 +0000 UTC m=+467.748973728" watchObservedRunningTime="2026-07-08 23:10:45.99070581 +0000 UTC m=+467.751200955" Jul 08 23:10:50.249585 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:50.249528 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:50.250198 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:50.249628 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:50.254447 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:50.254421 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:50.998965 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:50.998935 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b4b74587c-ccgb8" Jul 08 23:10:51.047192 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:51.047156 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-857c96864c-dfpcf"] Jul 08 23:10:53.963367 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:53.963337 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tchdl" Jul 08 23:10:56.979156 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:10:56.979128 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v6wdk" Jul 08 23:11:10.605591 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:10.605547 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v6wdk"] Jul 08 23:11:10.606041 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:10.605815 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v6wdk" podUID="5cb2a2a0-b852-4d71-b5b6-fa360d2279ef" containerName="manager" containerID="cri-o://b01aafbde3fc34736a780cc195f94f6db50d10e33141c01be8b0c511e601d597" gracePeriod=2 Jul 08 23:11:10.619226 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:10.619200 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v6wdk"] Jul 08 23:11:10.627146 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:11:10.627109 2548 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"5cb2a2a0-b852-4d71-b5b6-fa360d2279ef\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-v6wdk\": pods \"limitador-operator-controller-manager-859d7b78bd-v6wdk\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v6wdk" Jul 08 23:11:10.853815 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:10.853793 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v6wdk" Jul 08 23:11:10.948263 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:10.948183 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncdtr\" (UniqueName: \"kubernetes.io/projected/5cb2a2a0-b852-4d71-b5b6-fa360d2279ef-kube-api-access-ncdtr\") pod \"5cb2a2a0-b852-4d71-b5b6-fa360d2279ef\" (UID: \"5cb2a2a0-b852-4d71-b5b6-fa360d2279ef\") " Jul 08 23:11:10.950201 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:10.950172 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cb2a2a0-b852-4d71-b5b6-fa360d2279ef-kube-api-access-ncdtr" (OuterVolumeSpecName: "kube-api-access-ncdtr") pod "5cb2a2a0-b852-4d71-b5b6-fa360d2279ef" (UID: "5cb2a2a0-b852-4d71-b5b6-fa360d2279ef"). InnerVolumeSpecName "kube-api-access-ncdtr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:11:11.049407 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:11.049377 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncdtr\" (UniqueName: \"kubernetes.io/projected/5cb2a2a0-b852-4d71-b5b6-fa360d2279ef-kube-api-access-ncdtr\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:11:11.094891 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:11.094862 2548 generic.go:356] "Generic (PLEG): container finished" podID="5cb2a2a0-b852-4d71-b5b6-fa360d2279ef" containerID="b01aafbde3fc34736a780cc195f94f6db50d10e33141c01be8b0c511e601d597" exitCode=0 Jul 08 23:11:11.095058 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:11.094910 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v6wdk" Jul 08 23:11:11.095058 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:11.094913 2548 scope.go:117] "RemoveContainer" containerID="b01aafbde3fc34736a780cc195f94f6db50d10e33141c01be8b0c511e601d597" Jul 08 23:11:11.103983 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:11.103960 2548 scope.go:117] "RemoveContainer" containerID="b01aafbde3fc34736a780cc195f94f6db50d10e33141c01be8b0c511e601d597" Jul 08 23:11:11.104245 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:11:11.104224 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b01aafbde3fc34736a780cc195f94f6db50d10e33141c01be8b0c511e601d597\": container with ID starting with b01aafbde3fc34736a780cc195f94f6db50d10e33141c01be8b0c511e601d597 not found: ID does not exist" containerID="b01aafbde3fc34736a780cc195f94f6db50d10e33141c01be8b0c511e601d597" Jul 08 23:11:11.104295 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:11.104253 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b01aafbde3fc34736a780cc195f94f6db50d10e33141c01be8b0c511e601d597"} err="failed to get container status \"b01aafbde3fc34736a780cc195f94f6db50d10e33141c01be8b0c511e601d597\": rpc error: code = NotFound desc = could not find container \"b01aafbde3fc34736a780cc195f94f6db50d10e33141c01be8b0c511e601d597\": container with ID starting with b01aafbde3fc34736a780cc195f94f6db50d10e33141c01be8b0c511e601d597 not found: ID does not exist" Jul 08 23:11:12.828160 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:12.828130 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cb2a2a0-b852-4d71-b5b6-fa360d2279ef" path="/var/lib/kubelet/pods/5cb2a2a0-b852-4d71-b5b6-fa360d2279ef/volumes" Jul 08 23:11:16.068441 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.068379 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-857c96864c-dfpcf" podUID="4b19d4a7-63ac-4dec-8337-b09907a31586" containerName="console" containerID="cri-o://a0a8cbc73db045ffab4051325d52798027b4fb70384b3b13730933e6c44e68f4" gracePeriod=15 Jul 08 23:11:16.318552 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.318494 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:11:16.396556 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.396526 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-console-config\") pod \"4b19d4a7-63ac-4dec-8337-b09907a31586\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " Jul 08 23:11:16.396556 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.396560 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-trusted-ca-bundle\") pod \"4b19d4a7-63ac-4dec-8337-b09907a31586\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " Jul 08 23:11:16.396777 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.396620 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-oauth-serving-cert\") pod \"4b19d4a7-63ac-4dec-8337-b09907a31586\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " Jul 08 23:11:16.397073 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.397043 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-service-ca\") pod \"4b19d4a7-63ac-4dec-8337-b09907a31586\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " Jul 08 23:11:16.397225 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.397153 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-console-config" (OuterVolumeSpecName: "console-config") pod "4b19d4a7-63ac-4dec-8337-b09907a31586" (UID: "4b19d4a7-63ac-4dec-8337-b09907a31586"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:11:16.397225 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.397180 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4b19d4a7-63ac-4dec-8337-b09907a31586-console-oauth-config\") pod \"4b19d4a7-63ac-4dec-8337-b09907a31586\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " Jul 08 23:11:16.397225 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.397217 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4b19d4a7-63ac-4dec-8337-b09907a31586-console-serving-cert\") pod \"4b19d4a7-63ac-4dec-8337-b09907a31586\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " Jul 08 23:11:16.397398 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.397285 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zp5qm\" (UniqueName: \"kubernetes.io/projected/4b19d4a7-63ac-4dec-8337-b09907a31586-kube-api-access-zp5qm\") pod \"4b19d4a7-63ac-4dec-8337-b09907a31586\" (UID: \"4b19d4a7-63ac-4dec-8337-b09907a31586\") " Jul 08 23:11:16.397452 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.397401 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4b19d4a7-63ac-4dec-8337-b09907a31586" (UID: "4b19d4a7-63ac-4dec-8337-b09907a31586"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:11:16.397452 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.397428 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4b19d4a7-63ac-4dec-8337-b09907a31586" (UID: "4b19d4a7-63ac-4dec-8337-b09907a31586"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:11:16.397566 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.397461 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-service-ca" (OuterVolumeSpecName: "service-ca") pod "4b19d4a7-63ac-4dec-8337-b09907a31586" (UID: "4b19d4a7-63ac-4dec-8337-b09907a31586"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:11:16.398801 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.398458 2548 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-console-config\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:11:16.398801 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.398490 2548 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-trusted-ca-bundle\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:11:16.398801 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.398513 2548 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-oauth-serving-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:11:16.398801 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.398527 2548 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4b19d4a7-63ac-4dec-8337-b09907a31586-service-ca\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:11:16.400446 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.400062 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b19d4a7-63ac-4dec-8337-b09907a31586-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4b19d4a7-63ac-4dec-8337-b09907a31586" (UID: "4b19d4a7-63ac-4dec-8337-b09907a31586"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 23:11:16.400552 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.400508 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b19d4a7-63ac-4dec-8337-b09907a31586-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4b19d4a7-63ac-4dec-8337-b09907a31586" (UID: "4b19d4a7-63ac-4dec-8337-b09907a31586"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 23:11:16.406235 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.406209 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b19d4a7-63ac-4dec-8337-b09907a31586-kube-api-access-zp5qm" (OuterVolumeSpecName: "kube-api-access-zp5qm") pod "4b19d4a7-63ac-4dec-8337-b09907a31586" (UID: "4b19d4a7-63ac-4dec-8337-b09907a31586"). InnerVolumeSpecName "kube-api-access-zp5qm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:11:16.499406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.499362 2548 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4b19d4a7-63ac-4dec-8337-b09907a31586-console-oauth-config\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:11:16.499406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.499394 2548 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4b19d4a7-63ac-4dec-8337-b09907a31586-console-serving-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:11:16.499406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:16.499404 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zp5qm\" (UniqueName: \"kubernetes.io/projected/4b19d4a7-63ac-4dec-8337-b09907a31586-kube-api-access-zp5qm\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:11:17.127481 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:17.127394 2548 generic.go:356] "Generic (PLEG): container finished" podID="4b19d4a7-63ac-4dec-8337-b09907a31586" containerID="a0a8cbc73db045ffab4051325d52798027b4fb70384b3b13730933e6c44e68f4" exitCode=0 Jul 08 23:11:17.127481 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:17.127432 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857c96864c-dfpcf" event={"ID":"4b19d4a7-63ac-4dec-8337-b09907a31586","Type":"ContainerDied","Data":"a0a8cbc73db045ffab4051325d52798027b4fb70384b3b13730933e6c44e68f4"} Jul 08 23:11:17.127481 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:17.127460 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857c96864c-dfpcf" Jul 08 23:11:17.127481 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:17.127477 2548 scope.go:117] "RemoveContainer" containerID="a0a8cbc73db045ffab4051325d52798027b4fb70384b3b13730933e6c44e68f4" Jul 08 23:11:17.128073 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:17.127467 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857c96864c-dfpcf" event={"ID":"4b19d4a7-63ac-4dec-8337-b09907a31586","Type":"ContainerDied","Data":"e0017f84956e87deb28274ff3ec25bd224d0a0e6fb1bcbf8574fcd63e2a16035"} Jul 08 23:11:17.136206 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:17.136186 2548 scope.go:117] "RemoveContainer" containerID="a0a8cbc73db045ffab4051325d52798027b4fb70384b3b13730933e6c44e68f4" Jul 08 23:11:17.136494 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:11:17.136474 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0a8cbc73db045ffab4051325d52798027b4fb70384b3b13730933e6c44e68f4\": container with ID starting with a0a8cbc73db045ffab4051325d52798027b4fb70384b3b13730933e6c44e68f4 not found: ID does not exist" containerID="a0a8cbc73db045ffab4051325d52798027b4fb70384b3b13730933e6c44e68f4" Jul 08 23:11:17.136584 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:17.136503 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0a8cbc73db045ffab4051325d52798027b4fb70384b3b13730933e6c44e68f4"} err="failed to get container status \"a0a8cbc73db045ffab4051325d52798027b4fb70384b3b13730933e6c44e68f4\": rpc error: code = NotFound desc = could not find container \"a0a8cbc73db045ffab4051325d52798027b4fb70384b3b13730933e6c44e68f4\": container with ID starting with a0a8cbc73db045ffab4051325d52798027b4fb70384b3b13730933e6c44e68f4 not found: ID does not exist" Jul 08 23:11:17.165157 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:17.165126 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-857c96864c-dfpcf"] Jul 08 23:11:17.173245 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:17.173220 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-857c96864c-dfpcf"] Jul 08 23:11:18.828499 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:18.828459 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b19d4a7-63ac-4dec-8337-b09907a31586" path="/var/lib/kubelet/pods/4b19d4a7-63ac-4dec-8337-b09907a31586/volumes" Jul 08 23:11:35.318735 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:35.318699 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c"] Jul 08 23:11:35.319179 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:35.319022 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b19d4a7-63ac-4dec-8337-b09907a31586" containerName="console" Jul 08 23:11:35.319179 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:35.319033 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b19d4a7-63ac-4dec-8337-b09907a31586" containerName="console" Jul 08 23:11:35.319179 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:35.319055 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cb2a2a0-b852-4d71-b5b6-fa360d2279ef" containerName="manager" Jul 08 23:11:35.319179 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:35.319061 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cb2a2a0-b852-4d71-b5b6-fa360d2279ef" containerName="manager" Jul 08 23:11:35.319179 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:35.319115 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cb2a2a0-b852-4d71-b5b6-fa360d2279ef" containerName="manager" Jul 08 23:11:35.319179 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:35.319125 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b19d4a7-63ac-4dec-8337-b09907a31586" containerName="console" Jul 08 23:11:35.322065 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:35.322041 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c" Jul 08 23:11:35.325104 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:35.325087 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-b6jvt\"" Jul 08 23:11:35.333757 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:35.333736 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c"] Jul 08 23:11:35.351369 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:35.351326 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h99cf\" (UniqueName: \"kubernetes.io/projected/a34143bc-8166-4d9d-9a96-bd628af5112b-kube-api-access-h99cf\") pod \"kuadrant-operator-controller-manager-86878c769f-8l29c\" (UID: \"a34143bc-8166-4d9d-9a96-bd628af5112b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c" Jul 08 23:11:35.351578 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:35.351551 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a34143bc-8166-4d9d-9a96-bd628af5112b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8l29c\" (UID: \"a34143bc-8166-4d9d-9a96-bd628af5112b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c" Jul 08 23:11:35.452455 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:35.452414 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h99cf\" (UniqueName: \"kubernetes.io/projected/a34143bc-8166-4d9d-9a96-bd628af5112b-kube-api-access-h99cf\") pod \"kuadrant-operator-controller-manager-86878c769f-8l29c\" (UID: \"a34143bc-8166-4d9d-9a96-bd628af5112b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c" Jul 08 23:11:35.452455 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:35.452454 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a34143bc-8166-4d9d-9a96-bd628af5112b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8l29c\" (UID: \"a34143bc-8166-4d9d-9a96-bd628af5112b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c" Jul 08 23:11:35.452837 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:35.452820 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a34143bc-8166-4d9d-9a96-bd628af5112b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8l29c\" (UID: \"a34143bc-8166-4d9d-9a96-bd628af5112b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c" Jul 08 23:11:35.468417 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:35.468382 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h99cf\" (UniqueName: \"kubernetes.io/projected/a34143bc-8166-4d9d-9a96-bd628af5112b-kube-api-access-h99cf\") pod \"kuadrant-operator-controller-manager-86878c769f-8l29c\" (UID: \"a34143bc-8166-4d9d-9a96-bd628af5112b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c" Jul 08 23:11:35.631970 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:35.631887 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c" Jul 08 23:11:35.764939 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:35.764915 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c"] Jul 08 23:11:35.766957 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:11:35.766928 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda34143bc_8166_4d9d_9a96_bd628af5112b.slice/crio-5666a18202da9d63f3fdc25ff8fc53bfdd2dd1555e8cce04924409b650733447 WatchSource:0}: Error finding container 5666a18202da9d63f3fdc25ff8fc53bfdd2dd1555e8cce04924409b650733447: Status 404 returned error can't find the container with id 5666a18202da9d63f3fdc25ff8fc53bfdd2dd1555e8cce04924409b650733447 Jul 08 23:11:36.221367 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:36.221329 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c" event={"ID":"a34143bc-8166-4d9d-9a96-bd628af5112b","Type":"ContainerStarted","Data":"5666a18202da9d63f3fdc25ff8fc53bfdd2dd1555e8cce04924409b650733447"} Jul 08 23:11:41.250297 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:41.250256 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c" event={"ID":"a34143bc-8166-4d9d-9a96-bd628af5112b","Type":"ContainerStarted","Data":"5727e53d0901ab2fee642caafcaab9029f943eb56d7243da66bb7a521d6b0fb2"} Jul 08 23:11:41.250757 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:41.250369 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c" Jul 08 23:11:41.270566 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:41.270511 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c" podStartSLOduration=1.3370798210000001 podStartE2EDuration="6.270497658s" totalImagesPullingTime="4.933417837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:11:41.268270137 +0000 UTC m=+523.028765193" watchObservedRunningTime="2026-07-08 23:11:41.270497658 +0000 UTC m=+523.030992712" Jul 08 23:11:51.285284 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.285211 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl"] Jul 08 23:11:51.288844 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.288824 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.291489 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.291468 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-m5hc6\"" Jul 08 23:11:51.299738 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.299715 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl"] Jul 08 23:11:51.389000 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.388963 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.389000 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.389004 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.389549 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.389025 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.389549 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.389049 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.389549 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.389092 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.389549 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.389109 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66dms\" (UniqueName: \"kubernetes.io/projected/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-kube-api-access-66dms\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.389549 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.389225 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.389549 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.389271 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.389549 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.389328 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.490398 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.490356 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.490398 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.490404 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.490700 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.490430 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.490700 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.490454 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.490700 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.490490 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.490700 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.490516 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66dms\" (UniqueName: \"kubernetes.io/projected/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-kube-api-access-66dms\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.490700 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.490589 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.490700 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.490656 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.490700 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.490686 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.491049 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.490848 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.491049 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.490909 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.491120 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.491065 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.491171 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.491145 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.491390 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.491365 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.492832 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.492809 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.493036 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.493017 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.498124 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.498105 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.498235 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.498218 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66dms\" (UniqueName: \"kubernetes.io/projected/a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c-kube-api-access-66dms\") pod \"maas-default-gateway-openshift-default-56654dcf7-qlwdl\" (UID: \"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.600148 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.600057 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:51.736062 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.736038 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl"] Jul 08 23:11:51.738010 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:11:51.737982 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6a80db1_8a77_4f08_ae4c_7ef02f7cb01c.slice/crio-c650d9fbd87691e6e380870cadda0e84b55f0d0c8b329f9fbf3814053e17af9d WatchSource:0}: Error finding container c650d9fbd87691e6e380870cadda0e84b55f0d0c8b329f9fbf3814053e17af9d: Status 404 returned error can't find the container with id c650d9fbd87691e6e380870cadda0e84b55f0d0c8b329f9fbf3814053e17af9d Jul 08 23:11:51.740047 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.740014 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:11:51.740126 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.740080 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:11:51.740126 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:51.740110 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:11:52.258373 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:52.258341 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c" Jul 08 23:11:52.311518 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:52.311479 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" event={"ID":"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c","Type":"ContainerStarted","Data":"049ec56ef6dcd060c3d44401deb3c17efd10fedcf6678be52d0469a759721671"} Jul 08 23:11:52.311993 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:52.311532 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" event={"ID":"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c","Type":"ContainerStarted","Data":"c650d9fbd87691e6e380870cadda0e84b55f0d0c8b329f9fbf3814053e17af9d"} Jul 08 23:11:52.337419 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:52.337370 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" podStartSLOduration=1.337353536 podStartE2EDuration="1.337353536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:11:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:11:52.33543047 +0000 UTC m=+534.095925528" watchObservedRunningTime="2026-07-08 23:11:52.337353536 +0000 UTC m=+534.097848590" Jul 08 23:11:52.600541 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:52.600455 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:52.605194 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:52.605163 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:53.317298 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:53.317260 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:11:53.318373 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:11:53.318354 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:12:11.565952 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:12:11.565919 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:12:11.570413 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:12:11.570392 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-nn9bm" Jul 08 23:12:11.572961 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:12:11.572937 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 08 23:12:11.573048 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:12:11.572986 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lh2pm\"" Jul 08 23:12:11.578484 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:12:11.578461 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:12:11.593703 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:12:11.593668 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:12:11.672093 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:12:11.672061 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8efd0329-b2bf-4129-ac63-c4c87a831472-config-file\") pod \"limitador-limitador-54d6d988d5-nn9bm\" (UID: \"8efd0329-b2bf-4129-ac63-c4c87a831472\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nn9bm" Jul 08 23:12:11.672267 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:12:11.672131 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qztrz\" (UniqueName: \"kubernetes.io/projected/8efd0329-b2bf-4129-ac63-c4c87a831472-kube-api-access-qztrz\") pod \"limitador-limitador-54d6d988d5-nn9bm\" (UID: \"8efd0329-b2bf-4129-ac63-c4c87a831472\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nn9bm" Jul 08 23:12:11.773566 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:12:11.773527 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qztrz\" (UniqueName: \"kubernetes.io/projected/8efd0329-b2bf-4129-ac63-c4c87a831472-kube-api-access-qztrz\") pod \"limitador-limitador-54d6d988d5-nn9bm\" (UID: \"8efd0329-b2bf-4129-ac63-c4c87a831472\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nn9bm" Jul 08 23:12:11.773759 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:12:11.773591 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8efd0329-b2bf-4129-ac63-c4c87a831472-config-file\") pod \"limitador-limitador-54d6d988d5-nn9bm\" (UID: \"8efd0329-b2bf-4129-ac63-c4c87a831472\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nn9bm" Jul 08 23:12:11.774179 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:12:11.774159 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8efd0329-b2bf-4129-ac63-c4c87a831472-config-file\") pod \"limitador-limitador-54d6d988d5-nn9bm\" (UID: \"8efd0329-b2bf-4129-ac63-c4c87a831472\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nn9bm" Jul 08 23:12:11.781846 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:12:11.781828 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qztrz\" (UniqueName: \"kubernetes.io/projected/8efd0329-b2bf-4129-ac63-c4c87a831472-kube-api-access-qztrz\") pod \"limitador-limitador-54d6d988d5-nn9bm\" (UID: \"8efd0329-b2bf-4129-ac63-c4c87a831472\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nn9bm" Jul 08 23:12:11.881693 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:12:11.881583 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-nn9bm" Jul 08 23:12:12.017565 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:12:12.017540 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:12:12.019439 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:12:12.019403 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8efd0329_b2bf_4129_ac63_c4c87a831472.slice/crio-1050a9e1c9c71100dfa25fa0791063a1a6bbcb5e0fa43856755ac41d7be7171c WatchSource:0}: Error finding container 1050a9e1c9c71100dfa25fa0791063a1a6bbcb5e0fa43856755ac41d7be7171c: Status 404 returned error can't find the container with id 1050a9e1c9c71100dfa25fa0791063a1a6bbcb5e0fa43856755ac41d7be7171c Jul 08 23:12:12.413876 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:12:12.413842 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-nn9bm" event={"ID":"8efd0329-b2bf-4129-ac63-c4c87a831472","Type":"ContainerStarted","Data":"1050a9e1c9c71100dfa25fa0791063a1a6bbcb5e0fa43856755ac41d7be7171c"} Jul 08 23:12:15.433066 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:12:15.433031 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-nn9bm" event={"ID":"8efd0329-b2bf-4129-ac63-c4c87a831472","Type":"ContainerStarted","Data":"08a710577f3aa79dbecc5674dd3eb629a36af0b706476bfb64e4c4d44f138e80"} Jul 08 23:12:15.433432 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:12:15.433103 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-nn9bm" Jul 08 23:12:15.451516 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:12:15.451466 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-nn9bm" podStartSLOduration=1.930146366 podStartE2EDuration="4.451451803s" totalImagesPullingTime="2.521305437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:12:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:12:15.449190934 +0000 UTC m=+557.209685989" watchObservedRunningTime="2026-07-08 23:12:15.451451803 +0000 UTC m=+557.211946856" Jul 08 23:12:26.444272 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:12:26.444240 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-nn9bm" Jul 08 23:13:25.931815 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:25.931696 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:13:33.832941 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:33.832905 2548 generic.go:356] "Generic (PLEG): container finished" podID="a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c" containerID="049ec56ef6dcd060c3d44401deb3c17efd10fedcf6678be52d0469a759721671" exitCode=0 Jul 08 23:13:33.833354 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:33.832980 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" event={"ID":"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c","Type":"ContainerDied","Data":"049ec56ef6dcd060c3d44401deb3c17efd10fedcf6678be52d0469a759721671"} Jul 08 23:13:33.833397 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:33.833356 2548 scope.go:117] "RemoveContainer" containerID="049ec56ef6dcd060c3d44401deb3c17efd10fedcf6678be52d0469a759721671" Jul 08 23:13:33.834318 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:33.834288 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:13:33.834377 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:33.834356 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:13:33.834414 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:33.834388 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:13:34.840060 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:34.840024 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" event={"ID":"a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c","Type":"ContainerStarted","Data":"61901d8167c8ab4544db02e452a09c48b7b2669814aa8da307be95e434d92af7"} Jul 08 23:13:35.601112 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:35.601075 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:13:35.602027 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:35.601999 2548 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-qlwdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 08 23:13:35.602183 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:35.602053 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" podUID="a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 08 23:13:36.601070 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:36.601033 2548 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-qlwdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 08 23:13:36.601446 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:36.601103 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" podUID="a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 08 23:13:37.601220 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:37.601179 2548 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-qlwdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 08 23:13:37.601657 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:37.601246 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" podUID="a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 08 23:13:38.317318 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:38.317277 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:13:38.600725 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:38.600642 2548 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-qlwdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 08 23:13:38.600725 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:38.600697 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" podUID="a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 08 23:13:39.604757 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:39.604729 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:13:39.605834 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:39.605811 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qlwdl" Jul 08 23:13:42.790155 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:42.790116 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:13:52.401872 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:13:52.401837 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:14:02.598763 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:14:02.598724 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:14:09.196952 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:14:09.196916 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:14:25.806830 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:14:25.806794 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:14:36.399368 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:14:36.399326 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:15:00.153037 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:15:00.152957 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725875-4jlr2"] Jul 08 23:15:00.156273 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:15:00.156257 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725875-4jlr2" Jul 08 23:15:00.159573 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:15:00.159553 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 23:15:00.160703 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:15:00.160689 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tztbd\"" Jul 08 23:15:00.160771 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:15:00.160694 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 23:15:00.165327 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:15:00.165304 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725875-4jlr2"] Jul 08 23:15:00.235053 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:15:00.235017 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdbj9\" (UniqueName: \"kubernetes.io/projected/7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7-kube-api-access-tdbj9\") pod \"maas-api-key-cleanup-29725875-4jlr2\" (UID: \"7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725875-4jlr2" Jul 08 23:15:00.335978 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:15:00.335941 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdbj9\" (UniqueName: \"kubernetes.io/projected/7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7-kube-api-access-tdbj9\") pod \"maas-api-key-cleanup-29725875-4jlr2\" (UID: \"7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725875-4jlr2" Jul 08 23:15:00.345123 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:15:00.345098 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdbj9\" (UniqueName: \"kubernetes.io/projected/7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7-kube-api-access-tdbj9\") pod \"maas-api-key-cleanup-29725875-4jlr2\" (UID: \"7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725875-4jlr2" Jul 08 23:15:00.467299 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:15:00.467217 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725875-4jlr2" Jul 08 23:15:00.600329 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:15:00.600297 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725875-4jlr2"] Jul 08 23:15:00.602105 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:15:00.602072 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a7514a5_671a_4f04_aa2e_a9eaaea3e2b7.slice/crio-fd7b47e4892246c918fc62f6aadea68275eb55b1990e420bc514ef977402f5c9 WatchSource:0}: Error finding container fd7b47e4892246c918fc62f6aadea68275eb55b1990e420bc514ef977402f5c9: Status 404 returned error can't find the container with id fd7b47e4892246c918fc62f6aadea68275eb55b1990e420bc514ef977402f5c9 Jul 08 23:15:00.603933 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:15:00.603914 2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 23:15:01.297802 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:15:01.297766 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725875-4jlr2" event={"ID":"7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7","Type":"ContainerStarted","Data":"fd7b47e4892246c918fc62f6aadea68275eb55b1990e420bc514ef977402f5c9"} Jul 08 23:15:04.314078 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:15:04.314042 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725875-4jlr2" event={"ID":"7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7","Type":"ContainerStarted","Data":"003c90de70edf930a59df2400f2a661e9f0f0393aed07f05fcbcf384c505a694"} Jul 08 23:15:04.329622 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:15:04.329557 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725875-4jlr2" podStartSLOduration=1.8232164690000001 podStartE2EDuration="4.329540798s" totalImagesPullingTime="2.506324329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:15:04.329330755 +0000 UTC m=+726.089825824" watchObservedRunningTime="2026-07-08 23:15:04.329540798 +0000 UTC m=+726.090035852" Jul 08 23:15:42.505342 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:15:42.505309 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:15:52.691296 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:15:52.691261 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:16:00.010957 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:00.010913 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725875-4jlr2"] Jul 08 23:16:00.011415 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:00.011119 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725875-4jlr2" podUID="7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7" containerName="cleanup" containerID="cri-o://003c90de70edf930a59df2400f2a661e9f0f0393aed07f05fcbcf384c505a694" gracePeriod=30 Jul 08 23:16:04.700487 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:04.700445 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:16:12.438253 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:12.438215 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:16:23.115237 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:23.115158 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:16:30.184007 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:30.183981 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725875-4jlr2" Jul 08 23:16:30.328727 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:30.328635 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdbj9\" (UniqueName: \"kubernetes.io/projected/7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7-kube-api-access-tdbj9\") pod \"7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7\" (UID: \"7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7\") " Jul 08 23:16:30.330833 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:30.330796 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7-kube-api-access-tdbj9" (OuterVolumeSpecName: "kube-api-access-tdbj9") pod "7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7" (UID: "7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7"). InnerVolumeSpecName "kube-api-access-tdbj9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:16:30.429856 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:30.429823 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdbj9\" (UniqueName: \"kubernetes.io/projected/7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7-kube-api-access-tdbj9\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:16:30.771414 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:30.771378 2548 generic.go:356] "Generic (PLEG): container finished" podID="7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7" containerID="003c90de70edf930a59df2400f2a661e9f0f0393aed07f05fcbcf384c505a694" exitCode=137 Jul 08 23:16:30.771635 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:30.771430 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725875-4jlr2" Jul 08 23:16:30.771635 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:30.771439 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725875-4jlr2" event={"ID":"7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7","Type":"ContainerDied","Data":"003c90de70edf930a59df2400f2a661e9f0f0393aed07f05fcbcf384c505a694"} Jul 08 23:16:30.771635 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:30.771482 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725875-4jlr2" event={"ID":"7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7","Type":"ContainerDied","Data":"fd7b47e4892246c918fc62f6aadea68275eb55b1990e420bc514ef977402f5c9"} Jul 08 23:16:30.771635 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:30.771503 2548 scope.go:117] "RemoveContainer" containerID="003c90de70edf930a59df2400f2a661e9f0f0393aed07f05fcbcf384c505a694" Jul 08 23:16:30.780202 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:30.780185 2548 scope.go:117] "RemoveContainer" containerID="003c90de70edf930a59df2400f2a661e9f0f0393aed07f05fcbcf384c505a694" Jul 08 23:16:30.780443 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:16:30.780424 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"003c90de70edf930a59df2400f2a661e9f0f0393aed07f05fcbcf384c505a694\": container with ID starting with 003c90de70edf930a59df2400f2a661e9f0f0393aed07f05fcbcf384c505a694 not found: ID does not exist" containerID="003c90de70edf930a59df2400f2a661e9f0f0393aed07f05fcbcf384c505a694" Jul 08 23:16:30.780492 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:30.780453 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"003c90de70edf930a59df2400f2a661e9f0f0393aed07f05fcbcf384c505a694"} err="failed to get container status \"003c90de70edf930a59df2400f2a661e9f0f0393aed07f05fcbcf384c505a694\": rpc error: code = NotFound desc = could not find container \"003c90de70edf930a59df2400f2a661e9f0f0393aed07f05fcbcf384c505a694\": container with ID starting with 003c90de70edf930a59df2400f2a661e9f0f0393aed07f05fcbcf384c505a694 not found: ID does not exist" Jul 08 23:16:30.793870 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:30.793845 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725875-4jlr2"] Jul 08 23:16:30.797790 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:30.797768 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725875-4jlr2"] Jul 08 23:16:30.829325 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:30.829292 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7" path="/var/lib/kubelet/pods/7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7/volumes" Jul 08 23:16:51.103381 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:16:51.103344 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:17:05.002382 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:17:05.002349 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:17:37.111265 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:17:37.111185 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:17:43.706663 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:17:43.706626 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:17:52.994786 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:17:52.994701 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:18:02.091629 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:18:02.091582 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:18:13.297937 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:18:13.297901 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:18:22.696359 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:18:22.696321 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:18:33.597029 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:18:33.596990 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:18:43.495144 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:18:43.495108 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:19:08.996280 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:19:08.996245 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:19:18.394737 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:19:18.394703 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:19:40.494747 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:19:40.494713 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:19:51.399144 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:19:51.399107 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:20:06.992053 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:20:06.992016 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:20:18.498506 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:20:18.498469 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:20:28.100133 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:20:28.100097 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:20:37.301040 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:20:37.301000 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:20:46.590211 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:20:46.590174 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:21:11.479141 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:21:11.479058 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c"] Jul 08 23:21:11.479700 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:21:11.479291 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c" podUID="a34143bc-8166-4d9d-9a96-bd628af5112b" containerName="manager" containerID="cri-o://5727e53d0901ab2fee642caafcaab9029f943eb56d7243da66bb7a521d6b0fb2" gracePeriod=10 Jul 08 23:21:14.211033 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:21:14.211004 2548 generic.go:356] "Generic (PLEG): container finished" podID="a34143bc-8166-4d9d-9a96-bd628af5112b" containerID="5727e53d0901ab2fee642caafcaab9029f943eb56d7243da66bb7a521d6b0fb2" exitCode=0 Jul 08 23:21:14.211362 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:21:14.211078 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c" event={"ID":"a34143bc-8166-4d9d-9a96-bd628af5112b","Type":"ContainerDied","Data":"5727e53d0901ab2fee642caafcaab9029f943eb56d7243da66bb7a521d6b0fb2"} Jul 08 23:21:14.332892 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:21:14.332864 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c" Jul 08 23:21:14.395871 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:21:14.395841 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a34143bc-8166-4d9d-9a96-bd628af5112b-extensions-socket-volume\") pod \"a34143bc-8166-4d9d-9a96-bd628af5112b\" (UID: \"a34143bc-8166-4d9d-9a96-bd628af5112b\") " Jul 08 23:21:14.395871 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:21:14.395880 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h99cf\" (UniqueName: \"kubernetes.io/projected/a34143bc-8166-4d9d-9a96-bd628af5112b-kube-api-access-h99cf\") pod \"a34143bc-8166-4d9d-9a96-bd628af5112b\" (UID: \"a34143bc-8166-4d9d-9a96-bd628af5112b\") " Jul 08 23:21:14.396257 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:21:14.396201 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a34143bc-8166-4d9d-9a96-bd628af5112b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a34143bc-8166-4d9d-9a96-bd628af5112b" (UID: "a34143bc-8166-4d9d-9a96-bd628af5112b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:21:14.398028 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:21:14.398004 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a34143bc-8166-4d9d-9a96-bd628af5112b-kube-api-access-h99cf" (OuterVolumeSpecName: "kube-api-access-h99cf") pod "a34143bc-8166-4d9d-9a96-bd628af5112b" (UID: "a34143bc-8166-4d9d-9a96-bd628af5112b"). InnerVolumeSpecName "kube-api-access-h99cf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:21:14.497352 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:21:14.497322 2548 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a34143bc-8166-4d9d-9a96-bd628af5112b-extensions-socket-volume\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:21:14.497352 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:21:14.497348 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h99cf\" (UniqueName: \"kubernetes.io/projected/a34143bc-8166-4d9d-9a96-bd628af5112b-kube-api-access-h99cf\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:21:15.217475 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:21:15.217387 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c" Jul 08 23:21:15.217475 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:21:15.217377 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c" event={"ID":"a34143bc-8166-4d9d-9a96-bd628af5112b","Type":"ContainerDied","Data":"5666a18202da9d63f3fdc25ff8fc53bfdd2dd1555e8cce04924409b650733447"} Jul 08 23:21:15.217475 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:21:15.217458 2548 scope.go:117] "RemoveContainer" containerID="5727e53d0901ab2fee642caafcaab9029f943eb56d7243da66bb7a521d6b0fb2" Jul 08 23:21:15.234935 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:21:15.234906 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c"] Jul 08 23:21:15.238768 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:21:15.238742 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8l29c"] Jul 08 23:21:16.829489 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:21:16.829458 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a34143bc-8166-4d9d-9a96-bd628af5112b" path="/var/lib/kubelet/pods/a34143bc-8166-4d9d-9a96-bd628af5112b/volumes" Jul 08 23:22:12.610898 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:12.610861 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvdb9"] Jul 08 23:22:12.611296 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:12.611182 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a34143bc-8166-4d9d-9a96-bd628af5112b" containerName="manager" Jul 08 23:22:12.611296 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:12.611193 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="a34143bc-8166-4d9d-9a96-bd628af5112b" containerName="manager" Jul 08 23:22:12.611296 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:12.611200 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7" containerName="cleanup" Jul 08 23:22:12.611296 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:12.611206 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7" containerName="cleanup" Jul 08 23:22:12.611296 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:12.611267 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a7514a5-671a-4f04-aa2e-a9eaaea3e2b7" containerName="cleanup" Jul 08 23:22:12.611296 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:12.611276 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="a34143bc-8166-4d9d-9a96-bd628af5112b" containerName="manager" Jul 08 23:22:12.614921 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:12.614903 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvdb9" Jul 08 23:22:12.617480 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:12.617446 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-b6jvt\"" Jul 08 23:22:12.625415 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:12.625392 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvdb9"] Jul 08 23:22:12.682000 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:12.681959 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74513fe2-178e-4104-a6d7-441a1ceaf02d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nvdb9\" (UID: \"74513fe2-178e-4104-a6d7-441a1ceaf02d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvdb9" Jul 08 23:22:12.682177 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:12.682023 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngsxz\" (UniqueName: \"kubernetes.io/projected/74513fe2-178e-4104-a6d7-441a1ceaf02d-kube-api-access-ngsxz\") pod \"kuadrant-operator-controller-manager-86878c769f-nvdb9\" (UID: \"74513fe2-178e-4104-a6d7-441a1ceaf02d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvdb9" Jul 08 23:22:12.783370 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:12.783326 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74513fe2-178e-4104-a6d7-441a1ceaf02d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nvdb9\" (UID: \"74513fe2-178e-4104-a6d7-441a1ceaf02d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvdb9" Jul 08 23:22:12.783537 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:12.783390 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngsxz\" (UniqueName: \"kubernetes.io/projected/74513fe2-178e-4104-a6d7-441a1ceaf02d-kube-api-access-ngsxz\") pod \"kuadrant-operator-controller-manager-86878c769f-nvdb9\" (UID: \"74513fe2-178e-4104-a6d7-441a1ceaf02d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvdb9" Jul 08 23:22:12.783849 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:12.783830 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74513fe2-178e-4104-a6d7-441a1ceaf02d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nvdb9\" (UID: \"74513fe2-178e-4104-a6d7-441a1ceaf02d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvdb9" Jul 08 23:22:12.791907 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:12.791884 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngsxz\" (UniqueName: \"kubernetes.io/projected/74513fe2-178e-4104-a6d7-441a1ceaf02d-kube-api-access-ngsxz\") pod \"kuadrant-operator-controller-manager-86878c769f-nvdb9\" (UID: \"74513fe2-178e-4104-a6d7-441a1ceaf02d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvdb9" Jul 08 23:22:12.924982 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:12.924887 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvdb9" Jul 08 23:22:13.060013 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:13.059924 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvdb9"] Jul 08 23:22:13.062586 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:22:13.062559 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74513fe2_178e_4104_a6d7_441a1ceaf02d.slice/crio-a2a88e2e8845114d6e82b36ed0f8ae5e05feb4858b89d2af7ef68eb70d0bfcba WatchSource:0}: Error finding container a2a88e2e8845114d6e82b36ed0f8ae5e05feb4858b89d2af7ef68eb70d0bfcba: Status 404 returned error can't find the container with id a2a88e2e8845114d6e82b36ed0f8ae5e05feb4858b89d2af7ef68eb70d0bfcba Jul 08 23:22:13.516196 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:13.516161 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvdb9" event={"ID":"74513fe2-178e-4104-a6d7-441a1ceaf02d","Type":"ContainerStarted","Data":"ffe07dfba421e02292333f978afb5c39c4f2bdf1f44eff74300fe27a36b2671c"} Jul 08 23:22:13.516380 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:13.516249 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvdb9" Jul 08 23:22:13.516380 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:13.516272 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvdb9" event={"ID":"74513fe2-178e-4104-a6d7-441a1ceaf02d","Type":"ContainerStarted","Data":"a2a88e2e8845114d6e82b36ed0f8ae5e05feb4858b89d2af7ef68eb70d0bfcba"} Jul 08 23:22:13.551788 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:13.551733 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvdb9" podStartSLOduration=1.551717165 podStartE2EDuration="1.551717165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:22:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:22:13.54956706 +0000 UTC m=+1155.310062151" watchObservedRunningTime="2026-07-08 23:22:13.551717165 +0000 UTC m=+1155.312212219" Jul 08 23:22:24.523564 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:24.523483 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvdb9" Jul 08 23:22:33.299038 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:33.298999 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:22:42.592056 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:42.592018 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:22:55.306243 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:22:55.306204 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:23:06.596073 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:23:06.596036 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:23:18.192140 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:23:18.192102 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:23:28.189322 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:23:28.189285 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:23:38.705237 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:23:38.705190 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:23:48.701685 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:23:48.701649 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:23:59.892944 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:23:59.892855 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:24:08.600331 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:24:08.600288 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:24:19.095992 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:24:19.095958 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:24:43.792941 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:24:43.792897 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:24:50.898974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:24:50.898939 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:25:01.095151 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:25:01.095114 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:25:10.990696 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:25:10.990657 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:25:20.599387 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:25:20.599304 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:25:32.695986 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:25:32.695949 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:25:46.901084 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:25:46.901033 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:25:55.900960 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:25:55.900925 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:26:09.898956 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:26:09.898915 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:26:19.110558 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:26:19.108295 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:26:31.997496 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:26:31.997460 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:26:41.798686 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:26:41.798645 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:26:50.794169 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:26:50.794086 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:27:03.198192 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:27:03.198154 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:27:13.699111 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:27:13.699075 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:27:22.604701 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:27:22.604666 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:27:34.298337 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:27:34.298301 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:27:45.609289 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:27:45.609254 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:27:58.096176 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:27:58.096140 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:28:06.795024 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:06.794985 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:28:28.395486 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:28.395411 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:28:33.248948 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.248914 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5"] Jul 08 23:28:33.252443 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.252420 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.255033 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.255012 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-fda58ddc-openshift-default-dockercfg-k49lw\"" Jul 08 23:28:33.271165 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.271131 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5"] Jul 08 23:28:33.299955 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.299925 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-workload-socket\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.300097 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.299963 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-podinfo\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.300097 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.299984 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-data\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.300097 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.300043 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-workload-certs\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.300097 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.300077 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc74f6a1-05ff-457e-977c-36c564237dcf-istiod-ca-cert\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.300236 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.300108 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scm5z\" (UniqueName: \"kubernetes.io/projected/bc74f6a1-05ff-457e-977c-36c564237dcf-kube-api-access-scm5z\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.300236 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.300135 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-credential-socket\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.300236 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.300157 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-token\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.300236 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.300180 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-envoy\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.400563 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.400527 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-podinfo\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.400754 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.400572 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-data\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.400754 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.400721 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-workload-certs\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.400843 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.400756 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc74f6a1-05ff-457e-977c-36c564237dcf-istiod-ca-cert\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.400843 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.400809 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scm5z\" (UniqueName: \"kubernetes.io/projected/bc74f6a1-05ff-457e-977c-36c564237dcf-kube-api-access-scm5z\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.400990 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.400842 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-credential-socket\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.400990 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.400869 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-token\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.400990 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.400911 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-envoy\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.400990 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.400957 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-data\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.401234 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.401005 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-workload-socket\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.401234 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.401062 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-workload-certs\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.401361 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.401254 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-credential-socket\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.401432 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.401412 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-workload-socket\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.401628 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.401580 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc74f6a1-05ff-457e-977c-36c564237dcf-istiod-ca-cert\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.403101 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.403079 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-podinfo\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.403180 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.403130 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-envoy\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.408782 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.408754 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scm5z\" (UniqueName: \"kubernetes.io/projected/bc74f6a1-05ff-457e-977c-36c564237dcf-kube-api-access-scm5z\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.409406 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.409385 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-token\") pod \"e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.563375 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.563287 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:33.700772 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.700736 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5"] Jul 08 23:28:33.702891 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:28:33.702866 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc74f6a1_05ff_457e_977c_36c564237dcf.slice/crio-1d84345ed96fdea5db95384ecd9a176b96629d160e17c4d685f2f5c41a1c9ba2 WatchSource:0}: Error finding container 1d84345ed96fdea5db95384ecd9a176b96629d160e17c4d685f2f5c41a1c9ba2: Status 404 returned error can't find the container with id 1d84345ed96fdea5db95384ecd9a176b96629d160e17c4d685f2f5c41a1c9ba2 Jul 08 23:28:33.705268 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.705236 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:28:33.705352 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.705301 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:28:33.705352 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:33.705333 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:28:34.455999 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:34.455942 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" event={"ID":"bc74f6a1-05ff-457e-977c-36c564237dcf","Type":"ContainerStarted","Data":"d83ff5dc9629da0fbd2c9a22e1de892e0451cbd85fe3b573319f7690f234f2af"} Jul 08 23:28:34.456459 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:34.456166 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" event={"ID":"bc74f6a1-05ff-457e-977c-36c564237dcf","Type":"ContainerStarted","Data":"1d84345ed96fdea5db95384ecd9a176b96629d160e17c4d685f2f5c41a1c9ba2"} Jul 08 23:28:34.483879 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:34.483825 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" podStartSLOduration=1.483803917 podStartE2EDuration="1.483803917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:28:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:28:34.481761925 +0000 UTC m=+1536.242256990" watchObservedRunningTime="2026-07-08 23:28:34.483803917 +0000 UTC m=+1536.244298971" Jul 08 23:28:34.563996 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:34.563957 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:34.569045 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:34.569018 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:35.461172 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:35.461141 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:35.462116 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:35.462098 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:37.896273 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:37.896227 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:28:39.616304 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:39.616265 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5"] Jul 08 23:28:39.619045 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:39.616510 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" podUID="bc74f6a1-05ff-457e-977c-36c564237dcf" containerName="istio-proxy" containerID="cri-o://d83ff5dc9629da0fbd2c9a22e1de892e0451cbd85fe3b573319f7690f234f2af" gracePeriod=30 Jul 08 23:28:45.453717 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.453682 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg"] Jul 08 23:28:45.458902 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.458876 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.461522 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.461500 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1c7e051c-openshift-default-dockercfg-h7689\"" Jul 08 23:28:45.471287 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.471257 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg"] Jul 08 23:28:45.502402 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.502371 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-workload-socket\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.502402 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.502416 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-podinfo\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.502633 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.502449 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70997b1c-541b-4c85-bd20-d0bfe875702d-istiod-ca-cert\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.502633 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.502472 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-data\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.502633 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.502490 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt747\" (UniqueName: \"kubernetes.io/projected/70997b1c-541b-4c85-bd20-d0bfe875702d-kube-api-access-pt747\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.502633 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.502560 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-envoy\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.502633 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.502610 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-workload-certs\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.502818 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.502642 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-token\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.502818 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.502664 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-credential-socket\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.603065 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.603022 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-envoy\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.603261 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.603079 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-workload-certs\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.603261 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.603107 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-token\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.603261 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.603130 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-credential-socket\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.603261 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.603156 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-workload-socket\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.603261 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.603205 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-podinfo\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.603261 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.603245 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70997b1c-541b-4c85-bd20-d0bfe875702d-istiod-ca-cert\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.603737 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.603287 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-data\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.603737 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.603316 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt747\" (UniqueName: \"kubernetes.io/projected/70997b1c-541b-4c85-bd20-d0bfe875702d-kube-api-access-pt747\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.603737 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.603681 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-workload-certs\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.603737 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.603706 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-workload-socket\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.603958 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.603767 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-data\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.603958 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.603944 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-credential-socket\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.604224 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.604199 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70997b1c-541b-4c85-bd20-d0bfe875702d-istiod-ca-cert\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.605526 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.605498 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-envoy\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.605764 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.605747 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-podinfo\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.611310 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.611284 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-token\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.611503 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.611480 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt747\" (UniqueName: \"kubernetes.io/projected/70997b1c-541b-4c85-bd20-d0bfe875702d-kube-api-access-pt747\") pod \"e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.773665 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.773628 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:45.935425 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:28:45.935395 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70997b1c_541b_4c85_bd20_d0bfe875702d.slice/crio-d66a93e024fa155175834c6a0e8dce81d39c3c0d4599c397326cb435275505e5 WatchSource:0}: Error finding container d66a93e024fa155175834c6a0e8dce81d39c3c0d4599c397326cb435275505e5: Status 404 returned error can't find the container with id d66a93e024fa155175834c6a0e8dce81d39c3c0d4599c397326cb435275505e5 Jul 08 23:28:45.935573 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:45.935528 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg"] Jul 08 23:28:46.021458 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.021425 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:28:46.021540 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.021494 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:28:46.021540 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.021524 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:28:46.160996 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.160970 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:46.208269 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.208229 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-credential-socket\") pod \"bc74f6a1-05ff-457e-977c-36c564237dcf\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " Jul 08 23:28:46.208460 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.208284 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-podinfo\") pod \"bc74f6a1-05ff-457e-977c-36c564237dcf\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " Jul 08 23:28:46.208460 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.208331 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc74f6a1-05ff-457e-977c-36c564237dcf-istiod-ca-cert\") pod \"bc74f6a1-05ff-457e-977c-36c564237dcf\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " Jul 08 23:28:46.208460 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.208373 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-token\") pod \"bc74f6a1-05ff-457e-977c-36c564237dcf\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " Jul 08 23:28:46.208460 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.208456 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-data\") pod \"bc74f6a1-05ff-457e-977c-36c564237dcf\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " Jul 08 23:28:46.208694 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.208478 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-workload-certs\") pod \"bc74f6a1-05ff-457e-977c-36c564237dcf\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " Jul 08 23:28:46.208694 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.208519 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scm5z\" (UniqueName: \"kubernetes.io/projected/bc74f6a1-05ff-457e-977c-36c564237dcf-kube-api-access-scm5z\") pod \"bc74f6a1-05ff-457e-977c-36c564237dcf\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " Jul 08 23:28:46.208694 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.208555 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-workload-socket\") pod \"bc74f6a1-05ff-457e-977c-36c564237dcf\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " Jul 08 23:28:46.208694 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.208587 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-envoy\") pod \"bc74f6a1-05ff-457e-977c-36c564237dcf\" (UID: \"bc74f6a1-05ff-457e-977c-36c564237dcf\") " Jul 08 23:28:46.209483 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.209420 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bc74f6a1-05ff-457e-977c-36c564237dcf" (UID: "bc74f6a1-05ff-457e-977c-36c564237dcf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:28:46.210187 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.209920 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bc74f6a1-05ff-457e-977c-36c564237dcf" (UID: "bc74f6a1-05ff-457e-977c-36c564237dcf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:28:46.210286 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.210201 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-data" (OuterVolumeSpecName: "istio-data") pod "bc74f6a1-05ff-457e-977c-36c564237dcf" (UID: "bc74f6a1-05ff-457e-977c-36c564237dcf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:28:46.212508 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.211263 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bc74f6a1-05ff-457e-977c-36c564237dcf" (UID: "bc74f6a1-05ff-457e-977c-36c564237dcf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:28:46.212935 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.212907 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc74f6a1-05ff-457e-977c-36c564237dcf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bc74f6a1-05ff-457e-977c-36c564237dcf" (UID: "bc74f6a1-05ff-457e-977c-36c564237dcf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:28:46.224997 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.224966 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bc74f6a1-05ff-457e-977c-36c564237dcf" (UID: "bc74f6a1-05ff-457e-977c-36c564237dcf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:28:46.225824 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.225794 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bc74f6a1-05ff-457e-977c-36c564237dcf" (UID: "bc74f6a1-05ff-457e-977c-36c564237dcf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 23:28:46.225824 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.225805 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-token" (OuterVolumeSpecName: "istio-token") pod "bc74f6a1-05ff-457e-977c-36c564237dcf" (UID: "bc74f6a1-05ff-457e-977c-36c564237dcf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:28:46.226577 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.226555 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc74f6a1-05ff-457e-977c-36c564237dcf-kube-api-access-scm5z" (OuterVolumeSpecName: "kube-api-access-scm5z") pod "bc74f6a1-05ff-457e-977c-36c564237dcf" (UID: "bc74f6a1-05ff-457e-977c-36c564237dcf"). InnerVolumeSpecName "kube-api-access-scm5z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:28:46.310495 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.310406 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scm5z\" (UniqueName: \"kubernetes.io/projected/bc74f6a1-05ff-457e-977c-36c564237dcf-kube-api-access-scm5z\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:28:46.310495 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.310440 2548 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-workload-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:28:46.310495 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.310452 2548 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-envoy\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:28:46.310495 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.310460 2548 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-credential-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:28:46.310495 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.310468 2548 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-podinfo\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:28:46.310495 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.310476 2548 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc74f6a1-05ff-457e-977c-36c564237dcf-istiod-ca-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:28:46.310495 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.310483 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-token\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:28:46.310495 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.310491 2548 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-istio-data\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:28:46.310495 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.310498 2548 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc74f6a1-05ff-457e-977c-36c564237dcf-workload-certs\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:28:46.525229 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.525184 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" event={"ID":"70997b1c-541b-4c85-bd20-d0bfe875702d","Type":"ContainerStarted","Data":"84dad234295b7c8f1cf19a7eaccc835e1872ae103704eac86c97e2a90c713c51"} Jul 08 23:28:46.525703 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.525285 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" event={"ID":"70997b1c-541b-4c85-bd20-d0bfe875702d","Type":"ContainerStarted","Data":"d66a93e024fa155175834c6a0e8dce81d39c3c0d4599c397326cb435275505e5"} Jul 08 23:28:46.526510 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.526477 2548 generic.go:356] "Generic (PLEG): container finished" podID="bc74f6a1-05ff-457e-977c-36c564237dcf" containerID="d83ff5dc9629da0fbd2c9a22e1de892e0451cbd85fe3b573319f7690f234f2af" exitCode=0 Jul 08 23:28:46.526646 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.526531 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" Jul 08 23:28:46.526646 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.526554 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" event={"ID":"bc74f6a1-05ff-457e-977c-36c564237dcf","Type":"ContainerDied","Data":"d83ff5dc9629da0fbd2c9a22e1de892e0451cbd85fe3b573319f7690f234f2af"} Jul 08 23:28:46.526646 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.526622 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5" event={"ID":"bc74f6a1-05ff-457e-977c-36c564237dcf","Type":"ContainerDied","Data":"1d84345ed96fdea5db95384ecd9a176b96629d160e17c4d685f2f5c41a1c9ba2"} Jul 08 23:28:46.526646 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.526646 2548 scope.go:117] "RemoveContainer" containerID="d83ff5dc9629da0fbd2c9a22e1de892e0451cbd85fe3b573319f7690f234f2af" Jul 08 23:28:46.536807 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.536784 2548 scope.go:117] "RemoveContainer" containerID="d83ff5dc9629da0fbd2c9a22e1de892e0451cbd85fe3b573319f7690f234f2af" Jul 08 23:28:46.537102 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:28:46.537078 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d83ff5dc9629da0fbd2c9a22e1de892e0451cbd85fe3b573319f7690f234f2af\": container with ID starting with d83ff5dc9629da0fbd2c9a22e1de892e0451cbd85fe3b573319f7690f234f2af not found: ID does not exist" containerID="d83ff5dc9629da0fbd2c9a22e1de892e0451cbd85fe3b573319f7690f234f2af" Jul 08 23:28:46.537176 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.537115 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d83ff5dc9629da0fbd2c9a22e1de892e0451cbd85fe3b573319f7690f234f2af"} err="failed to get container status \"d83ff5dc9629da0fbd2c9a22e1de892e0451cbd85fe3b573319f7690f234f2af\": rpc error: code = NotFound desc = could not find container \"d83ff5dc9629da0fbd2c9a22e1de892e0451cbd85fe3b573319f7690f234f2af\": container with ID starting with d83ff5dc9629da0fbd2c9a22e1de892e0451cbd85fe3b573319f7690f234f2af not found: ID does not exist" Jul 08 23:28:46.553385 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.553328 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" podStartSLOduration=1.553312456 podStartE2EDuration="1.553312456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:28:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:28:46.552813814 +0000 UTC m=+1548.313308881" watchObservedRunningTime="2026-07-08 23:28:46.553312456 +0000 UTC m=+1548.313807510" Jul 08 23:28:46.576054 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.575972 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5"] Jul 08 23:28:46.588042 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.588011 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-fda58ddc-openshift-default-7bdf86dbf-frds5"] Jul 08 23:28:46.774448 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.774405 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:46.830347 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:46.830257 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc74f6a1-05ff-457e-977c-36c564237dcf" path="/var/lib/kubelet/pods/bc74f6a1-05ff-457e-977c-36c564237dcf/volumes" Jul 08 23:28:47.779330 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:47.779294 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:48.538362 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:48.538335 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:48.539197 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:48.539180 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:50.991592 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:50.991556 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:28:57.728908 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:57.728865 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg"] Jul 08 23:28:57.729847 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:57.729809 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" podUID="70997b1c-541b-4c85-bd20-d0bfe875702d" containerName="istio-proxy" containerID="cri-o://84dad234295b7c8f1cf19a7eaccc835e1872ae103704eac86c97e2a90c713c51" gracePeriod=30 Jul 08 23:28:58.993423 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:58.993398 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:59.028249 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.028169 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-envoy\") pod \"70997b1c-541b-4c85-bd20-d0bfe875702d\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " Jul 08 23:28:59.028249 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.028214 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-token\") pod \"70997b1c-541b-4c85-bd20-d0bfe875702d\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " Jul 08 23:28:59.028249 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.028247 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-workload-certs\") pod \"70997b1c-541b-4c85-bd20-d0bfe875702d\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " Jul 08 23:28:59.028636 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.028284 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-workload-socket\") pod \"70997b1c-541b-4c85-bd20-d0bfe875702d\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " Jul 08 23:28:59.028636 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.028326 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-data\") pod \"70997b1c-541b-4c85-bd20-d0bfe875702d\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " Jul 08 23:28:59.028636 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.028351 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pt747\" (UniqueName: \"kubernetes.io/projected/70997b1c-541b-4c85-bd20-d0bfe875702d-kube-api-access-pt747\") pod \"70997b1c-541b-4c85-bd20-d0bfe875702d\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " Jul 08 23:28:59.028636 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.028395 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70997b1c-541b-4c85-bd20-d0bfe875702d-istiod-ca-cert\") pod \"70997b1c-541b-4c85-bd20-d0bfe875702d\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " Jul 08 23:28:59.028636 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.028463 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-podinfo\") pod \"70997b1c-541b-4c85-bd20-d0bfe875702d\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " Jul 08 23:28:59.028636 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.028501 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-credential-socket\") pod \"70997b1c-541b-4c85-bd20-d0bfe875702d\" (UID: \"70997b1c-541b-4c85-bd20-d0bfe875702d\") " Jul 08 23:28:59.028636 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.028543 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "70997b1c-541b-4c85-bd20-d0bfe875702d" (UID: "70997b1c-541b-4c85-bd20-d0bfe875702d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:28:59.029378 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.028790 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "70997b1c-541b-4c85-bd20-d0bfe875702d" (UID: "70997b1c-541b-4c85-bd20-d0bfe875702d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:28:59.029378 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.028938 2548 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-workload-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:28:59.029378 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.028991 2548 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-workload-certs\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:28:59.029378 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.029012 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "70997b1c-541b-4c85-bd20-d0bfe875702d" (UID: "70997b1c-541b-4c85-bd20-d0bfe875702d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:28:59.029378 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.029268 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70997b1c-541b-4c85-bd20-d0bfe875702d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "70997b1c-541b-4c85-bd20-d0bfe875702d" (UID: "70997b1c-541b-4c85-bd20-d0bfe875702d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:28:59.029837 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.029803 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-data" (OuterVolumeSpecName: "istio-data") pod "70997b1c-541b-4c85-bd20-d0bfe875702d" (UID: "70997b1c-541b-4c85-bd20-d0bfe875702d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:28:59.031445 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.031412 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "70997b1c-541b-4c85-bd20-d0bfe875702d" (UID: "70997b1c-541b-4c85-bd20-d0bfe875702d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:28:59.031570 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.031548 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-token" (OuterVolumeSpecName: "istio-token") pod "70997b1c-541b-4c85-bd20-d0bfe875702d" (UID: "70997b1c-541b-4c85-bd20-d0bfe875702d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:28:59.031715 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.031696 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70997b1c-541b-4c85-bd20-d0bfe875702d-kube-api-access-pt747" (OuterVolumeSpecName: "kube-api-access-pt747") pod "70997b1c-541b-4c85-bd20-d0bfe875702d" (UID: "70997b1c-541b-4c85-bd20-d0bfe875702d"). InnerVolumeSpecName "kube-api-access-pt747". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:28:59.031894 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.031870 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "70997b1c-541b-4c85-bd20-d0bfe875702d" (UID: "70997b1c-541b-4c85-bd20-d0bfe875702d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 23:28:59.129997 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.129967 2548 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-data\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:28:59.129997 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.129992 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pt747\" (UniqueName: \"kubernetes.io/projected/70997b1c-541b-4c85-bd20-d0bfe875702d-kube-api-access-pt747\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:28:59.129997 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.130003 2548 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70997b1c-541b-4c85-bd20-d0bfe875702d-istiod-ca-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:28:59.130226 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.130012 2548 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-podinfo\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:28:59.130226 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.130020 2548 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-credential-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:28:59.130226 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.130028 2548 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-envoy\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:28:59.130226 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.130035 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70997b1c-541b-4c85-bd20-d0bfe875702d-istio-token\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:28:59.609015 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.608979 2548 generic.go:356] "Generic (PLEG): container finished" podID="70997b1c-541b-4c85-bd20-d0bfe875702d" containerID="84dad234295b7c8f1cf19a7eaccc835e1872ae103704eac86c97e2a90c713c51" exitCode=0 Jul 08 23:28:59.609203 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.609021 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" Jul 08 23:28:59.609203 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.609069 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" event={"ID":"70997b1c-541b-4c85-bd20-d0bfe875702d","Type":"ContainerDied","Data":"84dad234295b7c8f1cf19a7eaccc835e1872ae103704eac86c97e2a90c713c51"} Jul 08 23:28:59.609203 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.609123 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg" event={"ID":"70997b1c-541b-4c85-bd20-d0bfe875702d","Type":"ContainerDied","Data":"d66a93e024fa155175834c6a0e8dce81d39c3c0d4599c397326cb435275505e5"} Jul 08 23:28:59.609203 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.609151 2548 scope.go:117] "RemoveContainer" containerID="84dad234295b7c8f1cf19a7eaccc835e1872ae103704eac86c97e2a90c713c51" Jul 08 23:28:59.618344 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.618329 2548 scope.go:117] "RemoveContainer" containerID="84dad234295b7c8f1cf19a7eaccc835e1872ae103704eac86c97e2a90c713c51" Jul 08 23:28:59.618572 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:28:59.618557 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84dad234295b7c8f1cf19a7eaccc835e1872ae103704eac86c97e2a90c713c51\": container with ID starting with 84dad234295b7c8f1cf19a7eaccc835e1872ae103704eac86c97e2a90c713c51 not found: ID does not exist" containerID="84dad234295b7c8f1cf19a7eaccc835e1872ae103704eac86c97e2a90c713c51" Jul 08 23:28:59.618626 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.618581 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84dad234295b7c8f1cf19a7eaccc835e1872ae103704eac86c97e2a90c713c51"} err="failed to get container status \"84dad234295b7c8f1cf19a7eaccc835e1872ae103704eac86c97e2a90c713c51\": rpc error: code = NotFound desc = could not find container \"84dad234295b7c8f1cf19a7eaccc835e1872ae103704eac86c97e2a90c713c51\": container with ID starting with 84dad234295b7c8f1cf19a7eaccc835e1872ae103704eac86c97e2a90c713c51 not found: ID does not exist" Jul 08 23:28:59.630871 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.630845 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg"] Jul 08 23:28:59.634214 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:28:59.634194 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1c7e051c-openshift-default-54c79779bf-knjmg"] Jul 08 23:29:00.829145 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:00.829114 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70997b1c-541b-4c85-bd20-d0bfe875702d" path="/var/lib/kubelet/pods/70997b1c-541b-4c85-bd20-d0bfe875702d/volumes" Jul 08 23:29:02.693187 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:02.693139 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:29:03.731231 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.731200 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db"] Jul 08 23:29:03.731570 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.731557 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70997b1c-541b-4c85-bd20-d0bfe875702d" containerName="istio-proxy" Jul 08 23:29:03.731632 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.731572 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="70997b1c-541b-4c85-bd20-d0bfe875702d" containerName="istio-proxy" Jul 08 23:29:03.731632 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.731587 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc74f6a1-05ff-457e-977c-36c564237dcf" containerName="istio-proxy" Jul 08 23:29:03.731632 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.731592 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc74f6a1-05ff-457e-977c-36c564237dcf" containerName="istio-proxy" Jul 08 23:29:03.731763 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.731663 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc74f6a1-05ff-457e-977c-36c564237dcf" containerName="istio-proxy" Jul 08 23:29:03.731763 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.731682 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="70997b1c-541b-4c85-bd20-d0bfe875702d" containerName="istio-proxy" Jul 08 23:29:03.734282 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.734259 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.736854 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.736834 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-cb3326e4-openshift-default-dockercfg-nz86c\"" Jul 08 23:29:03.747546 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.747519 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db"] Jul 08 23:29:03.873872 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.873837 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-workload-socket\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.873872 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.873870 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-data\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.874148 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.873895 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-envoy\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.874148 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.874012 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-workload-certs\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.874148 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.874071 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istiod-ca-cert\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.874148 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.874110 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-podinfo\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.874340 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.874159 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9h2g\" (UniqueName: \"kubernetes.io/projected/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-kube-api-access-v9h2g\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.874340 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.874198 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-token\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.874340 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.874276 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-credential-socket\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.974967 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.974862 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9h2g\" (UniqueName: \"kubernetes.io/projected/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-kube-api-access-v9h2g\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.974967 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.974911 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-token\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.974967 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.974957 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-credential-socket\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.975255 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.974987 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-workload-socket\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.975255 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.975009 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-data\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.975255 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.975040 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-envoy\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.975255 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.975095 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-workload-certs\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.975255 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.975135 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istiod-ca-cert\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.975255 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.975171 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-podinfo\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.975558 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.975426 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-credential-socket\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.975558 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.975486 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-workload-socket\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.975558 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.975514 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-workload-certs\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.975734 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.975577 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-data\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.975938 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.975914 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istiod-ca-cert\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.977461 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.977435 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-envoy\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.977816 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.977799 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-podinfo\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.983195 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.983170 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9h2g\" (UniqueName: \"kubernetes.io/projected/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-kube-api-access-v9h2g\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:03.983525 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:03.983500 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-token\") pod \"e2e-derive-cb3326e4-openshift-default-74b786c77f-449db\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:04.046134 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:04.046105 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:04.183979 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:04.183955 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db"] Jul 08 23:29:04.189240 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:29:04.189203 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d00e0e7_c1ea_4331_9682_1dee846bc5fc.slice/crio-0a8c1f00fa78f25cf647e4e9d3057b7c02905b54c6b45c1e2fc39bfd3c3ab9af WatchSource:0}: Error finding container 0a8c1f00fa78f25cf647e4e9d3057b7c02905b54c6b45c1e2fc39bfd3c3ab9af: Status 404 returned error can't find the container with id 0a8c1f00fa78f25cf647e4e9d3057b7c02905b54c6b45c1e2fc39bfd3c3ab9af Jul 08 23:29:04.191103 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:04.191069 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:29:04.191207 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:04.191140 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:29:04.191207 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:04.191169 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:29:04.637551 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:04.637463 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" event={"ID":"5d00e0e7-c1ea-4331-9682-1dee846bc5fc","Type":"ContainerStarted","Data":"ba40712ba86077c3287769bdfae45afbb1b75823d310c0af47b1fc0e6246f507"} Jul 08 23:29:04.637551 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:04.637520 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" event={"ID":"5d00e0e7-c1ea-4331-9682-1dee846bc5fc","Type":"ContainerStarted","Data":"0a8c1f00fa78f25cf647e4e9d3057b7c02905b54c6b45c1e2fc39bfd3c3ab9af"} Jul 08 23:29:04.658398 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:04.658343 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" podStartSLOduration=1.658327213 podStartE2EDuration="1.658327213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:29:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:29:04.655933273 +0000 UTC m=+1566.416428339" watchObservedRunningTime="2026-07-08 23:29:04.658327213 +0000 UTC m=+1566.418822266" Jul 08 23:29:05.046976 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:05.046944 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:05.052767 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:05.052738 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:05.642841 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:05.642805 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:05.643884 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:05.643859 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:09.385729 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:09.385690 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db"] Jul 08 23:29:09.386139 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:09.385902 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" podUID="5d00e0e7-c1ea-4331-9682-1dee846bc5fc" containerName="istio-proxy" containerID="cri-o://ba40712ba86077c3287769bdfae45afbb1b75823d310c0af47b1fc0e6246f507" gracePeriod=30 Jul 08 23:29:15.844182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.844103 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:15.984158 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.984122 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-workload-socket\") pod \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " Jul 08 23:29:15.984158 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.984162 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9h2g\" (UniqueName: \"kubernetes.io/projected/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-kube-api-access-v9h2g\") pod \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " Jul 08 23:29:15.984422 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.984202 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-podinfo\") pod \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " Jul 08 23:29:15.984422 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.984230 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-credential-socket\") pod \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " Jul 08 23:29:15.984422 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.984269 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-token\") pod \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " Jul 08 23:29:15.984422 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.984351 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-envoy\") pod \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " Jul 08 23:29:15.984422 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.984384 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-workload-certs\") pod \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " Jul 08 23:29:15.984422 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.984419 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-data\") pod \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " Jul 08 23:29:15.984747 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.984445 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istiod-ca-cert\") pod \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\" (UID: \"5d00e0e7-c1ea-4331-9682-1dee846bc5fc\") " Jul 08 23:29:15.984747 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.984448 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5d00e0e7-c1ea-4331-9682-1dee846bc5fc" (UID: "5d00e0e7-c1ea-4331-9682-1dee846bc5fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:29:15.984747 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.984575 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5d00e0e7-c1ea-4331-9682-1dee846bc5fc" (UID: "5d00e0e7-c1ea-4331-9682-1dee846bc5fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:29:15.984747 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.984722 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5d00e0e7-c1ea-4331-9682-1dee846bc5fc" (UID: "5d00e0e7-c1ea-4331-9682-1dee846bc5fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:29:15.985018 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.984914 2548 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-workload-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:29:15.985018 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.984934 2548 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-credential-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:29:15.985018 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.984947 2548 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-workload-certs\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:29:15.985525 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.985278 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-data" (OuterVolumeSpecName: "istio-data") pod "5d00e0e7-c1ea-4331-9682-1dee846bc5fc" (UID: "5d00e0e7-c1ea-4331-9682-1dee846bc5fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:29:15.985525 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.985295 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5d00e0e7-c1ea-4331-9682-1dee846bc5fc" (UID: "5d00e0e7-c1ea-4331-9682-1dee846bc5fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:29:15.986693 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.986661 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5d00e0e7-c1ea-4331-9682-1dee846bc5fc" (UID: "5d00e0e7-c1ea-4331-9682-1dee846bc5fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:29:15.986980 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.986955 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-token" (OuterVolumeSpecName: "istio-token") pod "5d00e0e7-c1ea-4331-9682-1dee846bc5fc" (UID: "5d00e0e7-c1ea-4331-9682-1dee846bc5fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:29:15.987055 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.986961 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-kube-api-access-v9h2g" (OuterVolumeSpecName: "kube-api-access-v9h2g") pod "5d00e0e7-c1ea-4331-9682-1dee846bc5fc" (UID: "5d00e0e7-c1ea-4331-9682-1dee846bc5fc"). InnerVolumeSpecName "kube-api-access-v9h2g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:29:15.987101 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:15.987057 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5d00e0e7-c1ea-4331-9682-1dee846bc5fc" (UID: "5d00e0e7-c1ea-4331-9682-1dee846bc5fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 23:29:16.085451 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:16.085413 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9h2g\" (UniqueName: \"kubernetes.io/projected/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-kube-api-access-v9h2g\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:29:16.085451 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:16.085446 2548 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-podinfo\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:29:16.085693 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:16.085458 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-token\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:29:16.085693 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:16.085471 2548 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-envoy\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:29:16.085693 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:16.085481 2548 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istio-data\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:29:16.085693 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:16.085492 2548 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d00e0e7-c1ea-4331-9682-1dee846bc5fc-istiod-ca-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:29:16.705132 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:16.705100 2548 generic.go:356] "Generic (PLEG): container finished" podID="5d00e0e7-c1ea-4331-9682-1dee846bc5fc" containerID="ba40712ba86077c3287769bdfae45afbb1b75823d310c0af47b1fc0e6246f507" exitCode=0 Jul 08 23:29:16.705309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:16.705148 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" Jul 08 23:29:16.705309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:16.705154 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" event={"ID":"5d00e0e7-c1ea-4331-9682-1dee846bc5fc","Type":"ContainerDied","Data":"ba40712ba86077c3287769bdfae45afbb1b75823d310c0af47b1fc0e6246f507"} Jul 08 23:29:16.705309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:16.705197 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db" event={"ID":"5d00e0e7-c1ea-4331-9682-1dee846bc5fc","Type":"ContainerDied","Data":"0a8c1f00fa78f25cf647e4e9d3057b7c02905b54c6b45c1e2fc39bfd3c3ab9af"} Jul 08 23:29:16.705309 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:16.705219 2548 scope.go:117] "RemoveContainer" containerID="ba40712ba86077c3287769bdfae45afbb1b75823d310c0af47b1fc0e6246f507" Jul 08 23:29:16.714694 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:16.714677 2548 scope.go:117] "RemoveContainer" containerID="ba40712ba86077c3287769bdfae45afbb1b75823d310c0af47b1fc0e6246f507" Jul 08 23:29:16.714944 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:29:16.714925 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba40712ba86077c3287769bdfae45afbb1b75823d310c0af47b1fc0e6246f507\": container with ID starting with ba40712ba86077c3287769bdfae45afbb1b75823d310c0af47b1fc0e6246f507 not found: ID does not exist" containerID="ba40712ba86077c3287769bdfae45afbb1b75823d310c0af47b1fc0e6246f507" Jul 08 23:29:16.715016 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:16.714952 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba40712ba86077c3287769bdfae45afbb1b75823d310c0af47b1fc0e6246f507"} err="failed to get container status \"ba40712ba86077c3287769bdfae45afbb1b75823d310c0af47b1fc0e6246f507\": rpc error: code = NotFound desc = could not find container \"ba40712ba86077c3287769bdfae45afbb1b75823d310c0af47b1fc0e6246f507\": container with ID starting with ba40712ba86077c3287769bdfae45afbb1b75823d310c0af47b1fc0e6246f507 not found: ID does not exist" Jul 08 23:29:16.727473 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:16.727441 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db"] Jul 08 23:29:16.732232 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:16.732200 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-cb3326e4-openshift-default-74b786c77f-449db"] Jul 08 23:29:16.833737 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:29:16.830583 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d00e0e7-c1ea-4331-9682-1dee846bc5fc" path="/var/lib/kubelet/pods/5d00e0e7-c1ea-4331-9682-1dee846bc5fc/volumes" Jul 08 23:30:00.140630 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:00.140573 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725890-bb42z"] Jul 08 23:30:00.141102 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:00.141024 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d00e0e7-c1ea-4331-9682-1dee846bc5fc" containerName="istio-proxy" Jul 08 23:30:00.141102 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:00.141040 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d00e0e7-c1ea-4331-9682-1dee846bc5fc" containerName="istio-proxy" Jul 08 23:30:00.141205 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:00.141118 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d00e0e7-c1ea-4331-9682-1dee846bc5fc" containerName="istio-proxy" Jul 08 23:30:00.143249 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:00.143228 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725890-bb42z" Jul 08 23:30:00.146338 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:00.146310 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 23:30:00.147412 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:00.147389 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 23:30:00.147412 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:00.147400 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tztbd\"" Jul 08 23:30:00.153118 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:00.153099 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725890-bb42z"] Jul 08 23:30:00.270864 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:00.270821 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frgvn\" (UniqueName: \"kubernetes.io/projected/ca94f0d2-f1d9-437d-add5-018f14d21e4d-kube-api-access-frgvn\") pod \"maas-api-key-cleanup-29725890-bb42z\" (UID: \"ca94f0d2-f1d9-437d-add5-018f14d21e4d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725890-bb42z" Jul 08 23:30:00.371553 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:00.371515 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frgvn\" (UniqueName: \"kubernetes.io/projected/ca94f0d2-f1d9-437d-add5-018f14d21e4d-kube-api-access-frgvn\") pod \"maas-api-key-cleanup-29725890-bb42z\" (UID: \"ca94f0d2-f1d9-437d-add5-018f14d21e4d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725890-bb42z" Jul 08 23:30:00.380244 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:00.380213 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frgvn\" (UniqueName: \"kubernetes.io/projected/ca94f0d2-f1d9-437d-add5-018f14d21e4d-kube-api-access-frgvn\") pod \"maas-api-key-cleanup-29725890-bb42z\" (UID: \"ca94f0d2-f1d9-437d-add5-018f14d21e4d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725890-bb42z" Jul 08 23:30:00.455001 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:00.454907 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725890-bb42z" Jul 08 23:30:00.589625 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:00.589586 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725890-bb42z"] Jul 08 23:30:00.591517 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:30:00.591488 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca94f0d2_f1d9_437d_add5_018f14d21e4d.slice/crio-e128398c0227352517031b1e79008424aedf73ebdeaa0d50e52c246008719614 WatchSource:0}: Error finding container e128398c0227352517031b1e79008424aedf73ebdeaa0d50e52c246008719614: Status 404 returned error can't find the container with id e128398c0227352517031b1e79008424aedf73ebdeaa0d50e52c246008719614 Jul 08 23:30:00.926399 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:00.926362 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725890-bb42z" event={"ID":"ca94f0d2-f1d9-437d-add5-018f14d21e4d","Type":"ContainerStarted","Data":"ec17b6cf22c4a9d9b73323207234b947aace1bb9ad950e48d86eacab63efc475"} Jul 08 23:30:00.926620 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:00.926427 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725890-bb42z" event={"ID":"ca94f0d2-f1d9-437d-add5-018f14d21e4d","Type":"ContainerStarted","Data":"e128398c0227352517031b1e79008424aedf73ebdeaa0d50e52c246008719614"} Jul 08 23:30:00.941716 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:00.941514 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725890-bb42z" podStartSLOduration=0.941495219 podStartE2EDuration="941.495219ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:30:00.941241052 +0000 UTC m=+1622.701736108" watchObservedRunningTime="2026-07-08 23:30:00.941495219 +0000 UTC m=+1622.701990275" Jul 08 23:30:37.393936 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:37.393899 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:30:46.994763 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:46.994719 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:30:49.354094 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.354063 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s"] Jul 08 23:30:49.358114 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.358085 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.360706 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.360680 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-999d73b3-openshift-default-dockercfg-bdnnf\"" Jul 08 23:30:49.370435 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.370412 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s"] Jul 08 23:30:49.511517 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.511474 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrj2f\" (UniqueName: \"kubernetes.io/projected/9447e40b-f7f0-4286-ba32-924cbc4cdc47-kube-api-access-hrj2f\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.511721 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.511524 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-envoy\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.511721 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.511550 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-workload-certs\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.511721 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.511619 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-podinfo\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.511721 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.511653 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-token\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.511721 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.511678 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-data\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.511721 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.511707 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-credential-socket\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.511962 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.511778 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-workload-socket\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.511962 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.511806 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istiod-ca-cert\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.612863 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.612775 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-credential-socket\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.612863 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.612821 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-workload-socket\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.612863 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.612838 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istiod-ca-cert\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.613137 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.612871 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrj2f\" (UniqueName: \"kubernetes.io/projected/9447e40b-f7f0-4286-ba32-924cbc4cdc47-kube-api-access-hrj2f\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.613137 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.612894 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-envoy\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.613137 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.612914 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-workload-certs\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.613137 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.612941 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-podinfo\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.613137 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.612974 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-token\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.613137 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.613001 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-data\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.613446 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.613195 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-workload-socket\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.613446 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.613268 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-credential-socket\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.613446 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.613386 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-data\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.613583 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.613483 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-workload-certs\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.613659 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.613644 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istiod-ca-cert\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.615359 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.615335 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-envoy\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.615586 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.615566 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-podinfo\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.624168 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.624147 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-token\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.624695 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.624672 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrj2f\" (UniqueName: \"kubernetes.io/projected/9447e40b-f7f0-4286-ba32-924cbc4cdc47-kube-api-access-hrj2f\") pod \"e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.670041 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.670011 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:49.805712 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.805569 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s"] Jul 08 23:30:49.807982 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:30:49.807954 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9447e40b_f7f0_4286_ba32_924cbc4cdc47.slice/crio-632e3cc37e49dcc310835fe161cea63c3ea3808e89a1a60082a39bcf08d18c59 WatchSource:0}: Error finding container 632e3cc37e49dcc310835fe161cea63c3ea3808e89a1a60082a39bcf08d18c59: Status 404 returned error can't find the container with id 632e3cc37e49dcc310835fe161cea63c3ea3808e89a1a60082a39bcf08d18c59 Jul 08 23:30:49.810057 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.810029 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:30:49.810129 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.810089 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:30:49.810129 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:49.810118 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:30:50.193159 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:50.193072 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" event={"ID":"9447e40b-f7f0-4286-ba32-924cbc4cdc47","Type":"ContainerStarted","Data":"84ef9293fbff577d0e2512fb576f39a1132a887d29f867a0d15e276c33ebb679"} Jul 08 23:30:50.193159 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:50.193123 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" event={"ID":"9447e40b-f7f0-4286-ba32-924cbc4cdc47","Type":"ContainerStarted","Data":"632e3cc37e49dcc310835fe161cea63c3ea3808e89a1a60082a39bcf08d18c59"} Jul 08 23:30:50.213161 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:50.213114 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" podStartSLOduration=1.213098489 podStartE2EDuration="1.213098489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:30:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:30:50.210588957 +0000 UTC m=+1671.971084012" watchObservedRunningTime="2026-07-08 23:30:50.213098489 +0000 UTC m=+1671.973593601" Jul 08 23:30:50.670502 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:50.670474 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:50.675586 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:50.675559 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:51.198410 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:51.198375 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:51.199529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:51.199504 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:30:57.893322 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:30:57.893285 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:31:00.010735 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:00.010703 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725890-bb42z"] Jul 08 23:31:00.011137 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:00.010886 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725890-bb42z" podUID="ca94f0d2-f1d9-437d-add5-018f14d21e4d" containerName="cleanup" containerID="cri-o://ec17b6cf22c4a9d9b73323207234b947aace1bb9ad950e48d86eacab63efc475" gracePeriod=30 Jul 08 23:31:06.199877 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:06.199838 2548 patch_prober.go:28] interesting pod/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 08 23:31:06.200288 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:06.199895 2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" podUID="9447e40b-f7f0-4286-ba32-924cbc4cdc47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 08 23:31:06.297526 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:06.297494 2548 generic.go:356] "Generic (PLEG): container finished" podID="9447e40b-f7f0-4286-ba32-924cbc4cdc47" containerID="84ef9293fbff577d0e2512fb576f39a1132a887d29f867a0d15e276c33ebb679" exitCode=0 Jul 08 23:31:06.297711 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:06.297547 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" event={"ID":"9447e40b-f7f0-4286-ba32-924cbc4cdc47","Type":"ContainerDied","Data":"84ef9293fbff577d0e2512fb576f39a1132a887d29f867a0d15e276c33ebb679"} Jul 08 23:31:06.297974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:06.297959 2548 scope.go:117] "RemoveContainer" containerID="84ef9293fbff577d0e2512fb576f39a1132a887d29f867a0d15e276c33ebb679" Jul 08 23:31:06.298940 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:06.298914 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:31:06.299005 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:06.298981 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:31:06.299043 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:06.299020 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:31:07.304504 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:07.304475 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" event={"ID":"9447e40b-f7f0-4286-ba32-924cbc4cdc47","Type":"ContainerStarted","Data":"fcbba9b7a3cdc5ae57a86c26d58aafef10d65add68e654f0d6bbc5f7abe502b2"} Jul 08 23:31:07.671165 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:07.671050 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:31:07.671442 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:07.671275 2548 patch_prober.go:28] interesting pod/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 08 23:31:07.671442 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:07.671325 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" podUID="9447e40b-f7f0-4286-ba32-924cbc4cdc47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 08 23:31:08.670432 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:08.670395 2548 patch_prober.go:28] interesting pod/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 08 23:31:08.670937 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:08.670459 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" podUID="9447e40b-f7f0-4286-ba32-924cbc4cdc47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 08 23:31:08.702186 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:08.702153 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:31:09.671325 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:09.671287 2548 patch_prober.go:28] interesting pod/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 08 23:31:09.671824 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:09.671353 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" podUID="9447e40b-f7f0-4286-ba32-924cbc4cdc47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 08 23:31:10.204719 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:10.204683 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s"] Jul 08 23:31:10.204939 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:10.204914 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" podUID="9447e40b-f7f0-4286-ba32-924cbc4cdc47" containerName="istio-proxy" containerID="cri-o://fcbba9b7a3cdc5ae57a86c26d58aafef10d65add68e654f0d6bbc5f7abe502b2" gracePeriod=30 Jul 08 23:31:16.355350 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.355320 2548 generic.go:356] "Generic (PLEG): container finished" podID="9447e40b-f7f0-4286-ba32-924cbc4cdc47" containerID="fcbba9b7a3cdc5ae57a86c26d58aafef10d65add68e654f0d6bbc5f7abe502b2" exitCode=0 Jul 08 23:31:16.355789 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.355370 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" event={"ID":"9447e40b-f7f0-4286-ba32-924cbc4cdc47","Type":"ContainerDied","Data":"fcbba9b7a3cdc5ae57a86c26d58aafef10d65add68e654f0d6bbc5f7abe502b2"} Jul 08 23:31:16.355789 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.355416 2548 scope.go:117] "RemoveContainer" containerID="84ef9293fbff577d0e2512fb576f39a1132a887d29f867a0d15e276c33ebb679" Jul 08 23:31:16.459328 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.459283 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:31:16.544789 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.544613 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istiod-ca-cert\") pod \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " Jul 08 23:31:16.544789 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.544666 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrj2f\" (UniqueName: \"kubernetes.io/projected/9447e40b-f7f0-4286-ba32-924cbc4cdc47-kube-api-access-hrj2f\") pod \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " Jul 08 23:31:16.544789 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.544694 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-podinfo\") pod \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " Jul 08 23:31:16.544789 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.544745 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-envoy\") pod \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " Jul 08 23:31:16.544789 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.544784 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-credential-socket\") pod \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " Jul 08 23:31:16.545198 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.544810 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-workload-socket\") pod \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " Jul 08 23:31:16.545198 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.544862 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-token\") pod \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " Jul 08 23:31:16.545198 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.544953 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-workload-certs\") pod \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " Jul 08 23:31:16.545198 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.544976 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-data\") pod \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\" (UID: \"9447e40b-f7f0-4286-ba32-924cbc4cdc47\") " Jul 08 23:31:16.545198 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.545091 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9447e40b-f7f0-4286-ba32-924cbc4cdc47" (UID: "9447e40b-f7f0-4286-ba32-924cbc4cdc47"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:31:16.545531 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.545369 2548 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istiod-ca-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:31:16.545531 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.545463 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9447e40b-f7f0-4286-ba32-924cbc4cdc47" (UID: "9447e40b-f7f0-4286-ba32-924cbc4cdc47"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:31:16.545714 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.545678 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9447e40b-f7f0-4286-ba32-924cbc4cdc47" (UID: "9447e40b-f7f0-4286-ba32-924cbc4cdc47"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:31:16.545776 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.545745 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9447e40b-f7f0-4286-ba32-924cbc4cdc47" (UID: "9447e40b-f7f0-4286-ba32-924cbc4cdc47"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:31:16.546297 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.546274 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-data" (OuterVolumeSpecName: "istio-data") pod "9447e40b-f7f0-4286-ba32-924cbc4cdc47" (UID: "9447e40b-f7f0-4286-ba32-924cbc4cdc47"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:31:16.547670 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.547426 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9447e40b-f7f0-4286-ba32-924cbc4cdc47" (UID: "9447e40b-f7f0-4286-ba32-924cbc4cdc47"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:31:16.547670 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.547554 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9447e40b-f7f0-4286-ba32-924cbc4cdc47-kube-api-access-hrj2f" (OuterVolumeSpecName: "kube-api-access-hrj2f") pod "9447e40b-f7f0-4286-ba32-924cbc4cdc47" (UID: "9447e40b-f7f0-4286-ba32-924cbc4cdc47"). InnerVolumeSpecName "kube-api-access-hrj2f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:31:16.547858 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.547688 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9447e40b-f7f0-4286-ba32-924cbc4cdc47" (UID: "9447e40b-f7f0-4286-ba32-924cbc4cdc47"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 23:31:16.547938 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.547919 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-token" (OuterVolumeSpecName: "istio-token") pod "9447e40b-f7f0-4286-ba32-924cbc4cdc47" (UID: "9447e40b-f7f0-4286-ba32-924cbc4cdc47"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:31:16.646720 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.646689 2548 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-workload-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:31:16.646720 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.646715 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-token\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:31:16.646720 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.646724 2548 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-workload-certs\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:31:16.646720 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.646732 2548 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-data\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:31:16.647016 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.646741 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrj2f\" (UniqueName: \"kubernetes.io/projected/9447e40b-f7f0-4286-ba32-924cbc4cdc47-kube-api-access-hrj2f\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:31:16.647016 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.646750 2548 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-podinfo\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:31:16.647016 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.646760 2548 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-istio-envoy\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:31:16.647016 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:16.646768 2548 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9447e40b-f7f0-4286-ba32-924cbc4cdc47-credential-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:31:17.361657 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:17.361618 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" event={"ID":"9447e40b-f7f0-4286-ba32-924cbc4cdc47","Type":"ContainerDied","Data":"632e3cc37e49dcc310835fe161cea63c3ea3808e89a1a60082a39bcf08d18c59"} Jul 08 23:31:17.362102 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:17.361681 2548 scope.go:117] "RemoveContainer" containerID="fcbba9b7a3cdc5ae57a86c26d58aafef10d65add68e654f0d6bbc5f7abe502b2" Jul 08 23:31:17.362102 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:17.361628 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s" Jul 08 23:31:17.411058 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:17.411022 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s"] Jul 08 23:31:17.423234 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:17.423208 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-999d73b3-openshift-default-55bc468695-pxs4s"] Jul 08 23:31:18.829633 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:18.829574 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9447e40b-f7f0-4286-ba32-924cbc4cdc47" path="/var/lib/kubelet/pods/9447e40b-f7f0-4286-ba32-924cbc4cdc47/volumes" Jul 08 23:31:29.407403 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.407370 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7"] Jul 08 23:31:29.407910 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.407890 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9447e40b-f7f0-4286-ba32-924cbc4cdc47" containerName="istio-proxy" Jul 08 23:31:29.407987 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.407913 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="9447e40b-f7f0-4286-ba32-924cbc4cdc47" containerName="istio-proxy" Jul 08 23:31:29.407987 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.407930 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9447e40b-f7f0-4286-ba32-924cbc4cdc47" containerName="istio-proxy" Jul 08 23:31:29.407987 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.407938 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="9447e40b-f7f0-4286-ba32-924cbc4cdc47" containerName="istio-proxy" Jul 08 23:31:29.408132 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.408042 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="9447e40b-f7f0-4286-ba32-924cbc4cdc47" containerName="istio-proxy" Jul 08 23:31:29.408132 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.408057 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="9447e40b-f7f0-4286-ba32-924cbc4cdc47" containerName="istio-proxy" Jul 08 23:31:29.412508 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.412488 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.414990 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.414969 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9ebdfa23-openshift-default-dockercfg-4gdp8\"" Jul 08 23:31:29.425341 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.425317 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7"] Jul 08 23:31:29.454934 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.454902 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvmjl\" (UniqueName: \"kubernetes.io/projected/dba5a9a4-ea63-4107-bf1b-0758f999be26-kube-api-access-fvmjl\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.455122 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.454940 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-podinfo\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.455122 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.455024 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-workload-certs\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.455122 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.455070 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-credential-socket\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.455122 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.455116 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-data\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.455335 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.455150 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-envoy\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.455335 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.455247 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-workload-socket\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.455335 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.455274 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dba5a9a4-ea63-4107-bf1b-0758f999be26-istiod-ca-cert\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.455335 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.455308 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-token\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.555864 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.555829 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-workload-socket\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.556084 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.555873 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dba5a9a4-ea63-4107-bf1b-0758f999be26-istiod-ca-cert\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.556084 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.555903 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-token\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.556084 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.555933 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvmjl\" (UniqueName: \"kubernetes.io/projected/dba5a9a4-ea63-4107-bf1b-0758f999be26-kube-api-access-fvmjl\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.556084 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.555963 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-podinfo\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.556310 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.556095 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-workload-certs\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.556310 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.556170 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-credential-socket\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.556310 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.556227 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-data\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.556310 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.556267 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-envoy\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.556310 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.556290 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-workload-socket\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.556557 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.556459 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-workload-certs\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.556636 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.556581 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-credential-socket\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.556696 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.556653 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dba5a9a4-ea63-4107-bf1b-0758f999be26-istiod-ca-cert\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.556696 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.556681 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-data\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.558363 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.558340 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-envoy\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.558625 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.558577 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-podinfo\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.563882 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.563850 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-token\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.564052 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.564032 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvmjl\" (UniqueName: \"kubernetes.io/projected/dba5a9a4-ea63-4107-bf1b-0758f999be26-kube-api-access-fvmjl\") pod \"e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.724678 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.724559 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:29.858806 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.858779 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7"] Jul 08 23:31:29.860437 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:31:29.860410 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba5a9a4_ea63_4107_bf1b_0758f999be26.slice/crio-615333e19e3ff13aa2b4d32e698c1db9d61d9439eb8dff6c2fb244355b5f052c WatchSource:0}: Error finding container 615333e19e3ff13aa2b4d32e698c1db9d61d9439eb8dff6c2fb244355b5f052c: Status 404 returned error can't find the container with id 615333e19e3ff13aa2b4d32e698c1db9d61d9439eb8dff6c2fb244355b5f052c Jul 08 23:31:29.863019 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.862983 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:31:29.863125 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.863064 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:31:29.863125 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:29.863107 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:31:30.184271 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:30.184248 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725890-bb42z" Jul 08 23:31:30.264096 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:30.264071 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frgvn\" (UniqueName: \"kubernetes.io/projected/ca94f0d2-f1d9-437d-add5-018f14d21e4d-kube-api-access-frgvn\") pod \"ca94f0d2-f1d9-437d-add5-018f14d21e4d\" (UID: \"ca94f0d2-f1d9-437d-add5-018f14d21e4d\") " Jul 08 23:31:30.266151 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:30.266086 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca94f0d2-f1d9-437d-add5-018f14d21e4d-kube-api-access-frgvn" (OuterVolumeSpecName: "kube-api-access-frgvn") pod "ca94f0d2-f1d9-437d-add5-018f14d21e4d" (UID: "ca94f0d2-f1d9-437d-add5-018f14d21e4d"). InnerVolumeSpecName "kube-api-access-frgvn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:31:30.365561 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:30.365526 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frgvn\" (UniqueName: \"kubernetes.io/projected/ca94f0d2-f1d9-437d-add5-018f14d21e4d-kube-api-access-frgvn\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:31:30.431138 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:30.431105 2548 generic.go:356] "Generic (PLEG): container finished" podID="ca94f0d2-f1d9-437d-add5-018f14d21e4d" containerID="ec17b6cf22c4a9d9b73323207234b947aace1bb9ad950e48d86eacab63efc475" exitCode=137 Jul 08 23:31:30.431639 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:30.431149 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725890-bb42z" Jul 08 23:31:30.431639 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:30.431181 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725890-bb42z" event={"ID":"ca94f0d2-f1d9-437d-add5-018f14d21e4d","Type":"ContainerDied","Data":"ec17b6cf22c4a9d9b73323207234b947aace1bb9ad950e48d86eacab63efc475"} Jul 08 23:31:30.431639 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:30.431222 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725890-bb42z" event={"ID":"ca94f0d2-f1d9-437d-add5-018f14d21e4d","Type":"ContainerDied","Data":"e128398c0227352517031b1e79008424aedf73ebdeaa0d50e52c246008719614"} Jul 08 23:31:30.431639 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:30.431251 2548 scope.go:117] "RemoveContainer" containerID="ec17b6cf22c4a9d9b73323207234b947aace1bb9ad950e48d86eacab63efc475" Jul 08 23:31:30.432887 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:30.432839 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" event={"ID":"dba5a9a4-ea63-4107-bf1b-0758f999be26","Type":"ContainerStarted","Data":"9c61f048d9839cbaf7751d344d1d59669788e11b1a7aa5e572d35b360bb906c9"} Jul 08 23:31:30.432985 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:30.432905 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" event={"ID":"dba5a9a4-ea63-4107-bf1b-0758f999be26","Type":"ContainerStarted","Data":"615333e19e3ff13aa2b4d32e698c1db9d61d9439eb8dff6c2fb244355b5f052c"} Jul 08 23:31:30.440089 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:30.440071 2548 scope.go:117] "RemoveContainer" containerID="ec17b6cf22c4a9d9b73323207234b947aace1bb9ad950e48d86eacab63efc475" Jul 08 23:31:30.440347 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:31:30.440330 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec17b6cf22c4a9d9b73323207234b947aace1bb9ad950e48d86eacab63efc475\": container with ID starting with ec17b6cf22c4a9d9b73323207234b947aace1bb9ad950e48d86eacab63efc475 not found: ID does not exist" containerID="ec17b6cf22c4a9d9b73323207234b947aace1bb9ad950e48d86eacab63efc475" Jul 08 23:31:30.440396 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:30.440355 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec17b6cf22c4a9d9b73323207234b947aace1bb9ad950e48d86eacab63efc475"} err="failed to get container status \"ec17b6cf22c4a9d9b73323207234b947aace1bb9ad950e48d86eacab63efc475\": rpc error: code = NotFound desc = could not find container \"ec17b6cf22c4a9d9b73323207234b947aace1bb9ad950e48d86eacab63efc475\": container with ID starting with ec17b6cf22c4a9d9b73323207234b947aace1bb9ad950e48d86eacab63efc475 not found: ID does not exist" Jul 08 23:31:30.457374 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:30.457306 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" podStartSLOduration=1.457290565 podStartE2EDuration="1.457290565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:31:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:31:30.455835355 +0000 UTC m=+1712.216330408" watchObservedRunningTime="2026-07-08 23:31:30.457290565 +0000 UTC m=+1712.217785620" Jul 08 23:31:30.469390 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:30.469370 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725890-bb42z"] Jul 08 23:31:30.475134 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:30.475114 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725890-bb42z"] Jul 08 23:31:30.725434 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:30.725398 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:30.730168 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:30.730140 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:30.829344 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:30.829262 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca94f0d2-f1d9-437d-add5-018f14d21e4d" path="/var/lib/kubelet/pods/ca94f0d2-f1d9-437d-add5-018f14d21e4d/volumes" Jul 08 23:31:31.439467 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:31.439439 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:31.440651 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:31.440629 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:38.694220 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:38.694182 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:31:46.569928 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:46.569280 2548 patch_prober.go:28] interesting pod/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": read tcp 10.133.0.2:49944->10.133.0.49:15021: read: connection reset by peer" start-of-body= Jul 08 23:31:46.569928 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:46.569363 2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" podUID="dba5a9a4-ea63-4107-bf1b-0758f999be26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": read tcp 10.133.0.2:49944->10.133.0.49:15021: read: connection reset by peer" Jul 08 23:31:47.526776 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:47.526740 2548 generic.go:356] "Generic (PLEG): container finished" podID="dba5a9a4-ea63-4107-bf1b-0758f999be26" containerID="9c61f048d9839cbaf7751d344d1d59669788e11b1a7aa5e572d35b360bb906c9" exitCode=0 Jul 08 23:31:47.526964 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:47.526812 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" event={"ID":"dba5a9a4-ea63-4107-bf1b-0758f999be26","Type":"ContainerDied","Data":"9c61f048d9839cbaf7751d344d1d59669788e11b1a7aa5e572d35b360bb906c9"} Jul 08 23:31:47.527212 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:47.527194 2548 scope.go:117] "RemoveContainer" containerID="9c61f048d9839cbaf7751d344d1d59669788e11b1a7aa5e572d35b360bb906c9" Jul 08 23:31:47.528108 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:47.528077 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:31:47.528199 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:47.528148 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:31:47.528199 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:47.528179 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:31:48.534587 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:48.534550 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" event={"ID":"dba5a9a4-ea63-4107-bf1b-0758f999be26","Type":"ContainerStarted","Data":"da5dcb9840966b4e9f85d5a7d48a33982a2e79a38c7055fcc7508baab59be65b"} Jul 08 23:31:48.725304 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:48.725251 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:48.725623 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:48.725571 2548 patch_prober.go:28] interesting pod/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 08 23:31:48.725737 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:48.725661 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" podUID="dba5a9a4-ea63-4107-bf1b-0758f999be26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 08 23:31:49.725487 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:49.725450 2548 patch_prober.go:28] interesting pod/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 08 23:31:49.725926 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:49.725507 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" podUID="dba5a9a4-ea63-4107-bf1b-0758f999be26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 08 23:31:50.725152 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:50.725112 2548 patch_prober.go:28] interesting pod/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 08 23:31:50.725338 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:50.725171 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" podUID="dba5a9a4-ea63-4107-bf1b-0758f999be26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 08 23:31:51.724976 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:51.724935 2548 patch_prober.go:28] interesting pod/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 08 23:31:51.725389 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:51.724994 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" podUID="dba5a9a4-ea63-4107-bf1b-0758f999be26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 08 23:31:52.728945 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:52.728913 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:52.729320 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:52.729161 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:52.729810 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:52.729792 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:31:54.094207 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:31:54.094169 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:32:06.791512 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:32:06.791471 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:32:18.790256 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:32:18.790220 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:32:59.901240 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:32:59.897237 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:33:02.857308 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:02.857274 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7"] Jul 08 23:33:02.857720 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:02.857497 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" podUID="dba5a9a4-ea63-4107-bf1b-0758f999be26" containerName="istio-proxy" containerID="cri-o://da5dcb9840966b4e9f85d5a7d48a33982a2e79a38c7055fcc7508baab59be65b" gracePeriod=30 Jul 08 23:33:07.731980 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:07.731932 2548 patch_prober.go:28] interesting pod/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 23:33:07.732461 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:07.732028 2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" podUID="dba5a9a4-ea63-4107-bf1b-0758f999be26" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 23:33:08.104562 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.104533 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:33:08.226128 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.226098 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-data\") pod \"dba5a9a4-ea63-4107-bf1b-0758f999be26\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " Jul 08 23:33:08.226128 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.226137 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-workload-socket\") pod \"dba5a9a4-ea63-4107-bf1b-0758f999be26\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " Jul 08 23:33:08.226338 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.226158 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-podinfo\") pod \"dba5a9a4-ea63-4107-bf1b-0758f999be26\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " Jul 08 23:33:08.226338 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.226175 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-envoy\") pod \"dba5a9a4-ea63-4107-bf1b-0758f999be26\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " Jul 08 23:33:08.226338 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.226302 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvmjl\" (UniqueName: \"kubernetes.io/projected/dba5a9a4-ea63-4107-bf1b-0758f999be26-kube-api-access-fvmjl\") pod \"dba5a9a4-ea63-4107-bf1b-0758f999be26\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " Jul 08 23:33:08.226483 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.226356 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-credential-socket\") pod \"dba5a9a4-ea63-4107-bf1b-0758f999be26\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " Jul 08 23:33:08.226483 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.226382 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dba5a9a4-ea63-4107-bf1b-0758f999be26-istiod-ca-cert\") pod \"dba5a9a4-ea63-4107-bf1b-0758f999be26\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " Jul 08 23:33:08.226483 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.226428 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-token\") pod \"dba5a9a4-ea63-4107-bf1b-0758f999be26\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " Jul 08 23:33:08.226483 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.226466 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-workload-certs\") pod \"dba5a9a4-ea63-4107-bf1b-0758f999be26\" (UID: \"dba5a9a4-ea63-4107-bf1b-0758f999be26\") " Jul 08 23:33:08.226741 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.226424 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dba5a9a4-ea63-4107-bf1b-0758f999be26" (UID: "dba5a9a4-ea63-4107-bf1b-0758f999be26"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:33:08.226855 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.226833 2548 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-workload-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:33:08.226929 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.226631 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dba5a9a4-ea63-4107-bf1b-0758f999be26" (UID: "dba5a9a4-ea63-4107-bf1b-0758f999be26"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:33:08.226929 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.226866 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dba5a9a4-ea63-4107-bf1b-0758f999be26" (UID: "dba5a9a4-ea63-4107-bf1b-0758f999be26"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:33:08.227041 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.226934 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dba5a9a4-ea63-4107-bf1b-0758f999be26-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dba5a9a4-ea63-4107-bf1b-0758f999be26" (UID: "dba5a9a4-ea63-4107-bf1b-0758f999be26"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:33:08.227250 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.227231 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-data" (OuterVolumeSpecName: "istio-data") pod "dba5a9a4-ea63-4107-bf1b-0758f999be26" (UID: "dba5a9a4-ea63-4107-bf1b-0758f999be26"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:33:08.228394 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.228364 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dba5a9a4-ea63-4107-bf1b-0758f999be26" (UID: "dba5a9a4-ea63-4107-bf1b-0758f999be26"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 23:33:08.228506 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.228443 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dba5a9a4-ea63-4107-bf1b-0758f999be26-kube-api-access-fvmjl" (OuterVolumeSpecName: "kube-api-access-fvmjl") pod "dba5a9a4-ea63-4107-bf1b-0758f999be26" (UID: "dba5a9a4-ea63-4107-bf1b-0758f999be26"). InnerVolumeSpecName "kube-api-access-fvmjl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:33:08.228736 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.228719 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dba5a9a4-ea63-4107-bf1b-0758f999be26" (UID: "dba5a9a4-ea63-4107-bf1b-0758f999be26"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:33:08.228800 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.228775 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-token" (OuterVolumeSpecName: "istio-token") pod "dba5a9a4-ea63-4107-bf1b-0758f999be26" (UID: "dba5a9a4-ea63-4107-bf1b-0758f999be26"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:33:08.327752 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.327650 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvmjl\" (UniqueName: \"kubernetes.io/projected/dba5a9a4-ea63-4107-bf1b-0758f999be26-kube-api-access-fvmjl\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:33:08.327752 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.327693 2548 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-credential-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:33:08.327752 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.327703 2548 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dba5a9a4-ea63-4107-bf1b-0758f999be26-istiod-ca-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:33:08.327752 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.327711 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-token\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:33:08.327752 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.327720 2548 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-workload-certs\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:33:08.327752 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.327728 2548 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-data\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:33:08.327752 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.327735 2548 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-podinfo\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:33:08.327752 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.327742 2548 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dba5a9a4-ea63-4107-bf1b-0758f999be26-istio-envoy\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:33:08.955006 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.954969 2548 generic.go:356] "Generic (PLEG): container finished" podID="dba5a9a4-ea63-4107-bf1b-0758f999be26" containerID="da5dcb9840966b4e9f85d5a7d48a33982a2e79a38c7055fcc7508baab59be65b" exitCode=0 Jul 08 23:33:08.955451 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.955020 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" Jul 08 23:33:08.955451 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.955052 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" event={"ID":"dba5a9a4-ea63-4107-bf1b-0758f999be26","Type":"ContainerDied","Data":"da5dcb9840966b4e9f85d5a7d48a33982a2e79a38c7055fcc7508baab59be65b"} Jul 08 23:33:08.955451 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.955111 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7" event={"ID":"dba5a9a4-ea63-4107-bf1b-0758f999be26","Type":"ContainerDied","Data":"615333e19e3ff13aa2b4d32e698c1db9d61d9439eb8dff6c2fb244355b5f052c"} Jul 08 23:33:08.955451 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.955138 2548 scope.go:117] "RemoveContainer" containerID="da5dcb9840966b4e9f85d5a7d48a33982a2e79a38c7055fcc7508baab59be65b" Jul 08 23:33:08.964680 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.964658 2548 scope.go:117] "RemoveContainer" containerID="9c61f048d9839cbaf7751d344d1d59669788e11b1a7aa5e572d35b360bb906c9" Jul 08 23:33:08.973171 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.973146 2548 scope.go:117] "RemoveContainer" containerID="da5dcb9840966b4e9f85d5a7d48a33982a2e79a38c7055fcc7508baab59be65b" Jul 08 23:33:08.973248 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.973231 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7"] Jul 08 23:33:08.973514 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:33:08.973461 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da5dcb9840966b4e9f85d5a7d48a33982a2e79a38c7055fcc7508baab59be65b\": container with ID starting with da5dcb9840966b4e9f85d5a7d48a33982a2e79a38c7055fcc7508baab59be65b not found: ID does not exist" containerID="da5dcb9840966b4e9f85d5a7d48a33982a2e79a38c7055fcc7508baab59be65b" Jul 08 23:33:08.973639 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.973520 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da5dcb9840966b4e9f85d5a7d48a33982a2e79a38c7055fcc7508baab59be65b"} err="failed to get container status \"da5dcb9840966b4e9f85d5a7d48a33982a2e79a38c7055fcc7508baab59be65b\": rpc error: code = NotFound desc = could not find container \"da5dcb9840966b4e9f85d5a7d48a33982a2e79a38c7055fcc7508baab59be65b\": container with ID starting with da5dcb9840966b4e9f85d5a7d48a33982a2e79a38c7055fcc7508baab59be65b not found: ID does not exist" Jul 08 23:33:08.973639 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.973539 2548 scope.go:117] "RemoveContainer" containerID="9c61f048d9839cbaf7751d344d1d59669788e11b1a7aa5e572d35b360bb906c9" Jul 08 23:33:08.973913 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:33:08.973883 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c61f048d9839cbaf7751d344d1d59669788e11b1a7aa5e572d35b360bb906c9\": container with ID starting with 9c61f048d9839cbaf7751d344d1d59669788e11b1a7aa5e572d35b360bb906c9 not found: ID does not exist" containerID="9c61f048d9839cbaf7751d344d1d59669788e11b1a7aa5e572d35b360bb906c9" Jul 08 23:33:08.974032 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.973919 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c61f048d9839cbaf7751d344d1d59669788e11b1a7aa5e572d35b360bb906c9"} err="failed to get container status \"9c61f048d9839cbaf7751d344d1d59669788e11b1a7aa5e572d35b360bb906c9\": rpc error: code = NotFound desc = could not find container \"9c61f048d9839cbaf7751d344d1d59669788e11b1a7aa5e572d35b360bb906c9\": container with ID starting with 9c61f048d9839cbaf7751d344d1d59669788e11b1a7aa5e572d35b360bb906c9 not found: ID does not exist" Jul 08 23:33:08.977240 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:08.977221 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9ebdfa23-openshift-default-66bc5f684c-tfkp7"] Jul 08 23:33:10.828934 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:10.828899 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dba5a9a4-ea63-4107-bf1b-0758f999be26" path="/var/lib/kubelet/pods/dba5a9a4-ea63-4107-bf1b-0758f999be26/volumes" Jul 08 23:33:13.198933 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:13.198890 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:33:22.695571 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:22.695535 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:33:57.042248 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.042208 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29"] Jul 08 23:33:57.042839 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.042753 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dba5a9a4-ea63-4107-bf1b-0758f999be26" containerName="istio-proxy" Jul 08 23:33:57.042839 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.042774 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="dba5a9a4-ea63-4107-bf1b-0758f999be26" containerName="istio-proxy" Jul 08 23:33:57.042839 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.042788 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dba5a9a4-ea63-4107-bf1b-0758f999be26" containerName="istio-proxy" Jul 08 23:33:57.042839 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.042797 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="dba5a9a4-ea63-4107-bf1b-0758f999be26" containerName="istio-proxy" Jul 08 23:33:57.042839 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.042808 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca94f0d2-f1d9-437d-add5-018f14d21e4d" containerName="cleanup" Jul 08 23:33:57.042839 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.042817 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca94f0d2-f1d9-437d-add5-018f14d21e4d" containerName="cleanup" Jul 08 23:33:57.043156 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.042892 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="dba5a9a4-ea63-4107-bf1b-0758f999be26" containerName="istio-proxy" Jul 08 23:33:57.043156 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.042907 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca94f0d2-f1d9-437d-add5-018f14d21e4d" containerName="cleanup" Jul 08 23:33:57.046087 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.046066 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.049010 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.048991 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-96223d-openshift-default-dockercfg-96m96\"" Jul 08 23:33:57.057402 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.057375 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29"] Jul 08 23:33:57.155979 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.155948 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjjvj\" (UniqueName: \"kubernetes.io/projected/60676e38-5369-44c5-8eba-022cb04d3468-kube-api-access-pjjvj\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.156165 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.155985 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-workload-socket\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.156165 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.156012 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-workload-certs\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.156165 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.156073 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-istio-data\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.156165 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.156135 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-credential-socket\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.156165 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.156160 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-istio-envoy\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.156339 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.156196 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60676e38-5369-44c5-8eba-022cb04d3468-istio-token\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.156339 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.156254 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60676e38-5369-44c5-8eba-022cb04d3468-istio-podinfo\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.156339 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.156288 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60676e38-5369-44c5-8eba-022cb04d3468-istiod-ca-cert\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.257299 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.257261 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-credential-socket\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.257299 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.257298 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-istio-envoy\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.257575 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.257322 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60676e38-5369-44c5-8eba-022cb04d3468-istio-token\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.257575 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.257380 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60676e38-5369-44c5-8eba-022cb04d3468-istio-podinfo\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.257575 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.257408 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60676e38-5369-44c5-8eba-022cb04d3468-istiod-ca-cert\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.257773 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.257747 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjjvj\" (UniqueName: \"kubernetes.io/projected/60676e38-5369-44c5-8eba-022cb04d3468-kube-api-access-pjjvj\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.257831 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.257781 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-credential-socket\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.257831 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.257806 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-workload-socket\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.257932 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.257837 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-workload-certs\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.257932 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.257868 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-istio-data\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.258159 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.258137 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60676e38-5369-44c5-8eba-022cb04d3468-istiod-ca-cert\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.258159 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.258150 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-workload-socket\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.258277 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.258167 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-workload-certs\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.258277 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.258211 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-istio-data\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.260047 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.260013 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-istio-envoy\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.260143 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.260129 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60676e38-5369-44c5-8eba-022cb04d3468-istio-podinfo\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.265379 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.265347 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60676e38-5369-44c5-8eba-022cb04d3468-istio-token\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.265499 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.265475 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjjvj\" (UniqueName: \"kubernetes.io/projected/60676e38-5369-44c5-8eba-022cb04d3468-kube-api-access-pjjvj\") pod \"e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.356856 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.356769 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:57.501208 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.501178 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29"] Jul 08 23:33:57.502929 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:33:57.502901 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60676e38_5369_44c5_8eba_022cb04d3468.slice/crio-c0a88041194ac1a49074287b188d3d3042b0511df08f7056449707ba96ef27ff WatchSource:0}: Error finding container c0a88041194ac1a49074287b188d3d3042b0511df08f7056449707ba96ef27ff: Status 404 returned error can't find the container with id c0a88041194ac1a49074287b188d3d3042b0511df08f7056449707ba96ef27ff Jul 08 23:33:57.505134 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.505101 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:33:57.505233 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.505181 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:33:57.505298 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:57.505229 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:33:58.215265 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:58.215226 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" event={"ID":"60676e38-5369-44c5-8eba-022cb04d3468","Type":"ContainerStarted","Data":"8065e296ad00b0f48bcdba3a76e352bc1a4d32015742f96db109f34ff542a2ee"} Jul 08 23:33:58.215663 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:58.215284 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" event={"ID":"60676e38-5369-44c5-8eba-022cb04d3468","Type":"ContainerStarted","Data":"c0a88041194ac1a49074287b188d3d3042b0511df08f7056449707ba96ef27ff"} Jul 08 23:33:58.234399 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:58.234337 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" podStartSLOduration=1.234316467 podStartE2EDuration="1.234316467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:33:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:33:58.233541405 +0000 UTC m=+1859.994036472" watchObservedRunningTime="2026-07-08 23:33:58.234316467 +0000 UTC m=+1859.994811524" Jul 08 23:33:58.357155 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:58.357121 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:58.361611 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:58.361574 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:59.221349 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:59.221271 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:33:59.222138 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:33:59.222117 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:34:05.199200 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.199164 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7"] Jul 08 23:34:05.199923 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.199863 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="dba5a9a4-ea63-4107-bf1b-0758f999be26" containerName="istio-proxy" Jul 08 23:34:05.203117 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.203097 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.205687 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.205663 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-e9d26a-openshift-default-dockercfg-8w75x\"" Jul 08 23:34:05.213678 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.213656 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7"] Jul 08 23:34:05.333886 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.333849 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-workload-socket\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.333886 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.333894 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-istio-envoy\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.334157 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.333963 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-credential-socket\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.334157 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.334019 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-workload-certs\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.334157 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.334056 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b5476f-0128-4204-86ec-97d7396290af-istio-token\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.334157 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.334082 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvjl2\" (UniqueName: \"kubernetes.io/projected/a7b5476f-0128-4204-86ec-97d7396290af-kube-api-access-gvjl2\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.334157 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.334107 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-istio-data\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.334157 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.334134 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b5476f-0128-4204-86ec-97d7396290af-istiod-ca-cert\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.334416 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.334171 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b5476f-0128-4204-86ec-97d7396290af-istio-podinfo\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.435358 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.435317 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-istio-envoy\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.435557 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.435371 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-credential-socket\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.435557 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.435395 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-workload-certs\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.435557 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.435419 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b5476f-0128-4204-86ec-97d7396290af-istio-token\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.435557 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.435437 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvjl2\" (UniqueName: \"kubernetes.io/projected/a7b5476f-0128-4204-86ec-97d7396290af-kube-api-access-gvjl2\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.435557 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.435456 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-istio-data\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.435557 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.435476 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b5476f-0128-4204-86ec-97d7396290af-istiod-ca-cert\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.435557 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.435504 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b5476f-0128-4204-86ec-97d7396290af-istio-podinfo\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.435985 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.435625 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-workload-socket\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.435985 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.435946 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-credential-socket\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.436093 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.436029 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-workload-socket\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.436093 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.436044 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-istio-data\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.436093 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.436068 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-workload-certs\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.436316 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.436294 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b5476f-0128-4204-86ec-97d7396290af-istiod-ca-cert\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.437944 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.437916 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-istio-envoy\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.438059 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.438040 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b5476f-0128-4204-86ec-97d7396290af-istio-podinfo\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.443651 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.443587 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvjl2\" (UniqueName: \"kubernetes.io/projected/a7b5476f-0128-4204-86ec-97d7396290af-kube-api-access-gvjl2\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.443651 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.443617 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b5476f-0128-4204-86ec-97d7396290af-istio-token\") pod \"e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.513629 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.513576 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:05.650138 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.650110 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7"] Jul 08 23:34:05.652962 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:34:05.652936 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7b5476f_0128_4204_86ec_97d7396290af.slice/crio-5a93bfccf2c65d22bd9ebf5a78ed1a5c06d7aefc7380968291e6e3f3467d64b0 WatchSource:0}: Error finding container 5a93bfccf2c65d22bd9ebf5a78ed1a5c06d7aefc7380968291e6e3f3467d64b0: Status 404 returned error can't find the container with id 5a93bfccf2c65d22bd9ebf5a78ed1a5c06d7aefc7380968291e6e3f3467d64b0 Jul 08 23:34:05.655396 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.655366 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:34:05.655459 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.655436 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:34:05.655500 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:05.655466 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:34:06.260469 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:06.260433 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" event={"ID":"a7b5476f-0128-4204-86ec-97d7396290af","Type":"ContainerStarted","Data":"083330d65084ef44ecda0ef99cc2c4abd5fa570abe0d4b1caedae4d4b7ec3f43"} Jul 08 23:34:06.260891 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:06.260485 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" event={"ID":"a7b5476f-0128-4204-86ec-97d7396290af","Type":"ContainerStarted","Data":"5a93bfccf2c65d22bd9ebf5a78ed1a5c06d7aefc7380968291e6e3f3467d64b0"} Jul 08 23:34:06.282868 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:06.282821 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" podStartSLOduration=1.28280464 podStartE2EDuration="1.28280464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:34:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:34:06.279588719 +0000 UTC m=+1868.040083775" watchObservedRunningTime="2026-07-08 23:34:06.28280464 +0000 UTC m=+1868.043299693" Jul 08 23:34:06.513844 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:06.513751 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:06.518698 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:06.518675 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:07.265840 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:07.265807 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:07.266673 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:07.266655 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:07.896422 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:07.896390 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:34:15.263780 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:15.263734 2548 patch_prober.go:28] interesting pod/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 23:34:15.264293 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:15.263820 2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" podUID="60676e38-5369-44c5-8eba-022cb04d3468" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 23:34:16.326833 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:16.326794 2548 generic.go:356] "Generic (PLEG): container finished" podID="60676e38-5369-44c5-8eba-022cb04d3468" containerID="8065e296ad00b0f48bcdba3a76e352bc1a4d32015742f96db109f34ff542a2ee" exitCode=0 Jul 08 23:34:16.327303 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:16.326877 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" event={"ID":"60676e38-5369-44c5-8eba-022cb04d3468","Type":"ContainerDied","Data":"8065e296ad00b0f48bcdba3a76e352bc1a4d32015742f96db109f34ff542a2ee"} Jul 08 23:34:16.327303 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:16.327281 2548 scope.go:117] "RemoveContainer" containerID="8065e296ad00b0f48bcdba3a76e352bc1a4d32015742f96db109f34ff542a2ee" Jul 08 23:34:16.328294 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:16.328268 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:34:16.328417 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:16.328348 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:34:16.328417 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:16.328397 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:34:17.334242 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:17.334204 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" event={"ID":"60676e38-5369-44c5-8eba-022cb04d3468","Type":"ContainerStarted","Data":"9fb16272d0ea0121b9e4b688183f869cdd962145eaf97b529de5b4a6e2c72209"} Jul 08 23:34:17.357756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:17.357707 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:34:17.358196 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:17.358156 2548 patch_prober.go:28] interesting pod/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 08 23:34:17.358298 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:17.358233 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" podUID="60676e38-5369-44c5-8eba-022cb04d3468" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 08 23:34:18.357902 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:18.357865 2548 patch_prober.go:28] interesting pod/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 08 23:34:18.358884 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:18.358455 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" podUID="60676e38-5369-44c5-8eba-022cb04d3468" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 08 23:34:19.357614 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:19.357548 2548 patch_prober.go:28] interesting pod/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 08 23:34:19.357841 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:19.357644 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" podUID="60676e38-5369-44c5-8eba-022cb04d3468" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 08 23:34:20.357851 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:20.357747 2548 patch_prober.go:28] interesting pod/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 08 23:34:20.357851 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:20.357814 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" podUID="60676e38-5369-44c5-8eba-022cb04d3468" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 08 23:34:21.361409 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:21.361379 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:34:21.361870 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:21.361792 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:34:21.362588 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:21.362566 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:34:22.403159 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:22.403124 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:34:23.307764 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:23.307726 2548 patch_prober.go:28] interesting pod/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 23:34:23.307942 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:23.307796 2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" podUID="a7b5476f-0128-4204-86ec-97d7396290af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 23:34:24.380523 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:24.380434 2548 generic.go:356] "Generic (PLEG): container finished" podID="a7b5476f-0128-4204-86ec-97d7396290af" containerID="083330d65084ef44ecda0ef99cc2c4abd5fa570abe0d4b1caedae4d4b7ec3f43" exitCode=0 Jul 08 23:34:24.380523 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:24.380511 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" event={"ID":"a7b5476f-0128-4204-86ec-97d7396290af","Type":"ContainerDied","Data":"083330d65084ef44ecda0ef99cc2c4abd5fa570abe0d4b1caedae4d4b7ec3f43"} Jul 08 23:34:24.380992 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:24.380944 2548 scope.go:117] "RemoveContainer" containerID="083330d65084ef44ecda0ef99cc2c4abd5fa570abe0d4b1caedae4d4b7ec3f43" Jul 08 23:34:24.382108 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:24.382078 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:34:24.382181 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:24.382165 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:34:24.382224 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:24.382215 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:34:25.387660 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:25.387625 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" event={"ID":"a7b5476f-0128-4204-86ec-97d7396290af","Type":"ContainerStarted","Data":"5205cbf648e73da307a6ff470401dcb1f181cfefa6a0f0d7e8d70f4c5578d5ad"} Jul 08 23:34:25.514297 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:25.514261 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:25.514494 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:25.514473 2548 patch_prober.go:28] interesting pod/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 08 23:34:25.514567 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:25.514548 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" podUID="a7b5476f-0128-4204-86ec-97d7396290af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 08 23:34:26.514790 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:26.514753 2548 patch_prober.go:28] interesting pod/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 08 23:34:26.515161 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:26.514814 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" podUID="a7b5476f-0128-4204-86ec-97d7396290af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 08 23:34:27.514549 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:27.514513 2548 patch_prober.go:28] interesting pod/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 08 23:34:27.514761 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:27.514569 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" podUID="a7b5476f-0128-4204-86ec-97d7396290af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 08 23:34:28.515037 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:28.514998 2548 patch_prober.go:28] interesting pod/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 08 23:34:28.515504 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:28.515063 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" podUID="a7b5476f-0128-4204-86ec-97d7396290af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 08 23:34:29.518185 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:29.518156 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:29.518657 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:29.518584 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:29.519467 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:29.519435 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:34:35.189214 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:35.189181 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:34:50.898466 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:34:50.898432 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:35:08.997673 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:08.997628 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:35:38.949799 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:38.949761 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29"] Jul 08 23:35:38.950293 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:38.950031 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" podUID="60676e38-5369-44c5-8eba-022cb04d3468" containerName="istio-proxy" containerID="cri-o://9fb16272d0ea0121b9e4b688183f869cdd962145eaf97b529de5b4a6e2c72209" gracePeriod=30 Jul 08 23:35:43.191246 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:43.191204 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:35:44.213135 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.213072 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:35:44.363301 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.363263 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-istio-data\") pod \"60676e38-5369-44c5-8eba-022cb04d3468\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " Jul 08 23:35:44.363478 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.363321 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjjvj\" (UniqueName: \"kubernetes.io/projected/60676e38-5369-44c5-8eba-022cb04d3468-kube-api-access-pjjvj\") pod \"60676e38-5369-44c5-8eba-022cb04d3468\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " Jul 08 23:35:44.363478 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.363345 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-workload-socket\") pod \"60676e38-5369-44c5-8eba-022cb04d3468\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " Jul 08 23:35:44.363478 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.363367 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60676e38-5369-44c5-8eba-022cb04d3468-istiod-ca-cert\") pod \"60676e38-5369-44c5-8eba-022cb04d3468\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " Jul 08 23:35:44.363478 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.363432 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-istio-envoy\") pod \"60676e38-5369-44c5-8eba-022cb04d3468\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " Jul 08 23:35:44.363478 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.363472 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60676e38-5369-44c5-8eba-022cb04d3468-istio-podinfo\") pod \"60676e38-5369-44c5-8eba-022cb04d3468\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " Jul 08 23:35:44.363764 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.363508 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60676e38-5369-44c5-8eba-022cb04d3468-istio-token\") pod \"60676e38-5369-44c5-8eba-022cb04d3468\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " Jul 08 23:35:44.363764 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.363536 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-credential-socket\") pod \"60676e38-5369-44c5-8eba-022cb04d3468\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " Jul 08 23:35:44.363764 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.363565 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-workload-certs\") pod \"60676e38-5369-44c5-8eba-022cb04d3468\" (UID: \"60676e38-5369-44c5-8eba-022cb04d3468\") " Jul 08 23:35:44.363928 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.363765 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "60676e38-5369-44c5-8eba-022cb04d3468" (UID: "60676e38-5369-44c5-8eba-022cb04d3468"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:35:44.363928 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.363848 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60676e38-5369-44c5-8eba-022cb04d3468-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "60676e38-5369-44c5-8eba-022cb04d3468" (UID: "60676e38-5369-44c5-8eba-022cb04d3468"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:35:44.364044 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.363979 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "60676e38-5369-44c5-8eba-022cb04d3468" (UID: "60676e38-5369-44c5-8eba-022cb04d3468"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:35:44.364109 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.364081 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "60676e38-5369-44c5-8eba-022cb04d3468" (UID: "60676e38-5369-44c5-8eba-022cb04d3468"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:35:44.364167 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.364110 2548 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-workload-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:35:44.364167 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.364131 2548 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60676e38-5369-44c5-8eba-022cb04d3468-istiod-ca-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:35:44.364167 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.364140 2548 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-credential-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:35:44.364393 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.364365 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-istio-data" (OuterVolumeSpecName: "istio-data") pod "60676e38-5369-44c5-8eba-022cb04d3468" (UID: "60676e38-5369-44c5-8eba-022cb04d3468"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:35:44.365730 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.365696 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "60676e38-5369-44c5-8eba-022cb04d3468" (UID: "60676e38-5369-44c5-8eba-022cb04d3468"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:35:44.365848 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.365760 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60676e38-5369-44c5-8eba-022cb04d3468-kube-api-access-pjjvj" (OuterVolumeSpecName: "kube-api-access-pjjvj") pod "60676e38-5369-44c5-8eba-022cb04d3468" (UID: "60676e38-5369-44c5-8eba-022cb04d3468"). InnerVolumeSpecName "kube-api-access-pjjvj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:35:44.365848 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.365789 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60676e38-5369-44c5-8eba-022cb04d3468-istio-token" (OuterVolumeSpecName: "istio-token") pod "60676e38-5369-44c5-8eba-022cb04d3468" (UID: "60676e38-5369-44c5-8eba-022cb04d3468"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:35:44.365932 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.365914 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/60676e38-5369-44c5-8eba-022cb04d3468-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "60676e38-5369-44c5-8eba-022cb04d3468" (UID: "60676e38-5369-44c5-8eba-022cb04d3468"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 23:35:44.464909 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.464820 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60676e38-5369-44c5-8eba-022cb04d3468-istio-token\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:35:44.464909 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.464848 2548 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-workload-certs\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:35:44.464909 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.464858 2548 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-istio-data\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:35:44.464909 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.464866 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjjvj\" (UniqueName: \"kubernetes.io/projected/60676e38-5369-44c5-8eba-022cb04d3468-kube-api-access-pjjvj\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:35:44.464909 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.464877 2548 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60676e38-5369-44c5-8eba-022cb04d3468-istio-envoy\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:35:44.464909 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.464885 2548 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60676e38-5369-44c5-8eba-022cb04d3468-istio-podinfo\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:35:44.802336 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.802303 2548 generic.go:356] "Generic (PLEG): container finished" podID="60676e38-5369-44c5-8eba-022cb04d3468" containerID="9fb16272d0ea0121b9e4b688183f869cdd962145eaf97b529de5b4a6e2c72209" exitCode=0 Jul 08 23:35:44.802529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.802352 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" Jul 08 23:35:44.802529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.802387 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" event={"ID":"60676e38-5369-44c5-8eba-022cb04d3468","Type":"ContainerDied","Data":"9fb16272d0ea0121b9e4b688183f869cdd962145eaf97b529de5b4a6e2c72209"} Jul 08 23:35:44.802529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.802464 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29" event={"ID":"60676e38-5369-44c5-8eba-022cb04d3468","Type":"ContainerDied","Data":"c0a88041194ac1a49074287b188d3d3042b0511df08f7056449707ba96ef27ff"} Jul 08 23:35:44.802529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.802487 2548 scope.go:117] "RemoveContainer" containerID="9fb16272d0ea0121b9e4b688183f869cdd962145eaf97b529de5b4a6e2c72209" Jul 08 23:35:44.811820 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.811804 2548 scope.go:117] "RemoveContainer" containerID="8065e296ad00b0f48bcdba3a76e352bc1a4d32015742f96db109f34ff542a2ee" Jul 08 23:35:44.819999 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.819982 2548 scope.go:117] "RemoveContainer" containerID="9fb16272d0ea0121b9e4b688183f869cdd962145eaf97b529de5b4a6e2c72209" Jul 08 23:35:44.820257 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:35:44.820238 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fb16272d0ea0121b9e4b688183f869cdd962145eaf97b529de5b4a6e2c72209\": container with ID starting with 9fb16272d0ea0121b9e4b688183f869cdd962145eaf97b529de5b4a6e2c72209 not found: ID does not exist" containerID="9fb16272d0ea0121b9e4b688183f869cdd962145eaf97b529de5b4a6e2c72209" Jul 08 23:35:44.820304 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.820263 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fb16272d0ea0121b9e4b688183f869cdd962145eaf97b529de5b4a6e2c72209"} err="failed to get container status \"9fb16272d0ea0121b9e4b688183f869cdd962145eaf97b529de5b4a6e2c72209\": rpc error: code = NotFound desc = could not find container \"9fb16272d0ea0121b9e4b688183f869cdd962145eaf97b529de5b4a6e2c72209\": container with ID starting with 9fb16272d0ea0121b9e4b688183f869cdd962145eaf97b529de5b4a6e2c72209 not found: ID does not exist" Jul 08 23:35:44.820304 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.820286 2548 scope.go:117] "RemoveContainer" containerID="8065e296ad00b0f48bcdba3a76e352bc1a4d32015742f96db109f34ff542a2ee" Jul 08 23:35:44.820572 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:35:44.820541 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8065e296ad00b0f48bcdba3a76e352bc1a4d32015742f96db109f34ff542a2ee\": container with ID starting with 8065e296ad00b0f48bcdba3a76e352bc1a4d32015742f96db109f34ff542a2ee not found: ID does not exist" containerID="8065e296ad00b0f48bcdba3a76e352bc1a4d32015742f96db109f34ff542a2ee" Jul 08 23:35:44.820700 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.820571 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8065e296ad00b0f48bcdba3a76e352bc1a4d32015742f96db109f34ff542a2ee"} err="failed to get container status \"8065e296ad00b0f48bcdba3a76e352bc1a4d32015742f96db109f34ff542a2ee\": rpc error: code = NotFound desc = could not find container \"8065e296ad00b0f48bcdba3a76e352bc1a4d32015742f96db109f34ff542a2ee\": container with ID starting with 8065e296ad00b0f48bcdba3a76e352bc1a4d32015742f96db109f34ff542a2ee not found: ID does not exist" Jul 08 23:35:44.830214 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.830191 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29"] Jul 08 23:35:44.830330 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:44.830225 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-96223d-openshift-default-574c5c7cdf-nwp29"] Jul 08 23:35:46.055366 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.055335 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7"] Jul 08 23:35:46.055874 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.055562 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" podUID="a7b5476f-0128-4204-86ec-97d7396290af" containerName="istio-proxy" containerID="cri-o://5205cbf648e73da307a6ff470401dcb1f181cfefa6a0f0d7e8d70f4c5578d5ad" gracePeriod=30 Jul 08 23:35:46.829345 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.829311 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60676e38-5369-44c5-8eba-022cb04d3468" path="/var/lib/kubelet/pods/60676e38-5369-44c5-8eba-022cb04d3468/volumes" Jul 08 23:35:46.891486 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.891455 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w"] Jul 08 23:35:46.891853 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.891840 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60676e38-5369-44c5-8eba-022cb04d3468" containerName="istio-proxy" Jul 08 23:35:46.891903 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.891855 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="60676e38-5369-44c5-8eba-022cb04d3468" containerName="istio-proxy" Jul 08 23:35:46.891903 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.891868 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60676e38-5369-44c5-8eba-022cb04d3468" containerName="istio-proxy" Jul 08 23:35:46.891903 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.891874 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="60676e38-5369-44c5-8eba-022cb04d3468" containerName="istio-proxy" Jul 08 23:35:46.891999 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.891923 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="60676e38-5369-44c5-8eba-022cb04d3468" containerName="istio-proxy" Jul 08 23:35:46.891999 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.891933 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="60676e38-5369-44c5-8eba-022cb04d3468" containerName="istio-proxy" Jul 08 23:35:46.895018 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.895004 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:46.897682 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.897561 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-0a9b6b-openshift-default-dockercfg-rw9k4\"" Jul 08 23:35:46.906327 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.906301 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w"] Jul 08 23:35:46.987922 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.987392 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-credential-socket\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:46.987922 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.987471 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istiod-ca-cert\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:46.987922 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.987522 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-data\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:46.987922 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.987548 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-podinfo\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:46.987922 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.987586 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-token\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:46.987922 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.987646 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66kbz\" (UniqueName: \"kubernetes.io/projected/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-kube-api-access-66kbz\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:46.987922 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.987707 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-workload-certs\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:46.987922 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.987728 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-envoy\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:46.987922 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:46.987802 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-workload-socket\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.089025 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.088938 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-data\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.089025 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.088975 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-podinfo\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.089025 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.089014 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-token\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.089574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.089124 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66kbz\" (UniqueName: \"kubernetes.io/projected/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-kube-api-access-66kbz\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.089574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.089180 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-workload-certs\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.089574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.089211 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-envoy\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.089574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.089288 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-workload-socket\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.089574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.089331 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-credential-socket\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.089574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.089398 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-data\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.089574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.089401 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istiod-ca-cert\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.089574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.089556 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-workload-certs\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.090021 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.089687 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-credential-socket\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.090021 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.089742 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-workload-socket\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.090170 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.090146 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istiod-ca-cert\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.091418 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.091398 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-envoy\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.091683 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.091665 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-podinfo\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.096802 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.096780 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-token\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.097095 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.097072 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66kbz\" (UniqueName: \"kubernetes.io/projected/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-kube-api-access-66kbz\") pod \"e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.206285 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.206242 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:47.347541 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.347512 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w"] Jul 08 23:35:47.349331 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:35:47.349302 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9a81dd7_96fc_41dd_b82a_21b9ecf72700.slice/crio-f93a27f4c59351e46be8e647e21b22f88bfae3f2903491839949c3966a700cfe WatchSource:0}: Error finding container f93a27f4c59351e46be8e647e21b22f88bfae3f2903491839949c3966a700cfe: Status 404 returned error can't find the container with id f93a27f4c59351e46be8e647e21b22f88bfae3f2903491839949c3966a700cfe Jul 08 23:35:47.351848 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.351812 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:35:47.351939 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.351883 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:35:47.351939 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.351915 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:35:47.825009 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.824969 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" event={"ID":"d9a81dd7-96fc-41dd-b82a-21b9ecf72700","Type":"ContainerStarted","Data":"a0e05b6eec0e7565ae782ce5ddd24f5094dd0624130524e4fe7b6b5850a81da2"} Jul 08 23:35:47.825173 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.825054 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" event={"ID":"d9a81dd7-96fc-41dd-b82a-21b9ecf72700","Type":"ContainerStarted","Data":"f93a27f4c59351e46be8e647e21b22f88bfae3f2903491839949c3966a700cfe"} Jul 08 23:35:47.846974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:47.846925 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" podStartSLOduration=1.846910701 podStartE2EDuration="1.846910701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:35:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:35:47.843987188 +0000 UTC m=+1969.604482243" watchObservedRunningTime="2026-07-08 23:35:47.846910701 +0000 UTC m=+1969.607405754" Jul 08 23:35:48.207404 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:48.207312 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:48.212125 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:48.212096 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:48.829689 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:48.829657 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:48.830771 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:48.830749 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:35:51.313439 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.313414 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:35:51.431721 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.431625 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-istio-data\") pod \"a7b5476f-0128-4204-86ec-97d7396290af\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " Jul 08 23:35:51.431721 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.431676 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b5476f-0128-4204-86ec-97d7396290af-istio-token\") pod \"a7b5476f-0128-4204-86ec-97d7396290af\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " Jul 08 23:35:51.431721 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.431713 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b5476f-0128-4204-86ec-97d7396290af-istiod-ca-cert\") pod \"a7b5476f-0128-4204-86ec-97d7396290af\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " Jul 08 23:35:51.432062 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.431750 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-istio-envoy\") pod \"a7b5476f-0128-4204-86ec-97d7396290af\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " Jul 08 23:35:51.432062 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.431783 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-workload-certs\") pod \"a7b5476f-0128-4204-86ec-97d7396290af\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " Jul 08 23:35:51.432062 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.431823 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvjl2\" (UniqueName: \"kubernetes.io/projected/a7b5476f-0128-4204-86ec-97d7396290af-kube-api-access-gvjl2\") pod \"a7b5476f-0128-4204-86ec-97d7396290af\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " Jul 08 23:35:51.432062 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.431870 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-workload-socket\") pod \"a7b5476f-0128-4204-86ec-97d7396290af\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " Jul 08 23:35:51.432062 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.431894 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-credential-socket\") pod \"a7b5476f-0128-4204-86ec-97d7396290af\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " Jul 08 23:35:51.432062 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.431925 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b5476f-0128-4204-86ec-97d7396290af-istio-podinfo\") pod \"a7b5476f-0128-4204-86ec-97d7396290af\" (UID: \"a7b5476f-0128-4204-86ec-97d7396290af\") " Jul 08 23:35:51.432375 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.432128 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a7b5476f-0128-4204-86ec-97d7396290af" (UID: "a7b5476f-0128-4204-86ec-97d7396290af"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:35:51.432375 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.432144 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a7b5476f-0128-4204-86ec-97d7396290af" (UID: "a7b5476f-0128-4204-86ec-97d7396290af"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:35:51.432375 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.432262 2548 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-workload-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:35:51.432375 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.432283 2548 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-workload-certs\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:35:51.432375 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.432357 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a7b5476f-0128-4204-86ec-97d7396290af" (UID: "a7b5476f-0128-4204-86ec-97d7396290af"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:35:51.432651 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.432630 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b5476f-0128-4204-86ec-97d7396290af-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a7b5476f-0128-4204-86ec-97d7396290af" (UID: "a7b5476f-0128-4204-86ec-97d7396290af"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:35:51.432824 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.432798 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-istio-data" (OuterVolumeSpecName: "istio-data") pod "a7b5476f-0128-4204-86ec-97d7396290af" (UID: "a7b5476f-0128-4204-86ec-97d7396290af"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:35:51.434067 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.434040 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a7b5476f-0128-4204-86ec-97d7396290af-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a7b5476f-0128-4204-86ec-97d7396290af" (UID: "a7b5476f-0128-4204-86ec-97d7396290af"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 23:35:51.434067 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.434051 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7b5476f-0128-4204-86ec-97d7396290af-kube-api-access-gvjl2" (OuterVolumeSpecName: "kube-api-access-gvjl2") pod "a7b5476f-0128-4204-86ec-97d7396290af" (UID: "a7b5476f-0128-4204-86ec-97d7396290af"). InnerVolumeSpecName "kube-api-access-gvjl2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:35:51.434199 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.434078 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7b5476f-0128-4204-86ec-97d7396290af-istio-token" (OuterVolumeSpecName: "istio-token") pod "a7b5476f-0128-4204-86ec-97d7396290af" (UID: "a7b5476f-0128-4204-86ec-97d7396290af"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:35:51.434292 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.434278 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a7b5476f-0128-4204-86ec-97d7396290af" (UID: "a7b5476f-0128-4204-86ec-97d7396290af"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:35:51.533704 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.533670 2548 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-credential-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:35:51.533704 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.533696 2548 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b5476f-0128-4204-86ec-97d7396290af-istio-podinfo\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:35:51.533704 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.533705 2548 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-istio-data\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:35:51.533704 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.533713 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b5476f-0128-4204-86ec-97d7396290af-istio-token\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:35:51.533962 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.533724 2548 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b5476f-0128-4204-86ec-97d7396290af-istiod-ca-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:35:51.533962 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.533732 2548 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b5476f-0128-4204-86ec-97d7396290af-istio-envoy\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:35:51.533962 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.533741 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvjl2\" (UniqueName: \"kubernetes.io/projected/a7b5476f-0128-4204-86ec-97d7396290af-kube-api-access-gvjl2\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:35:51.849787 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.849745 2548 generic.go:356] "Generic (PLEG): container finished" podID="a7b5476f-0128-4204-86ec-97d7396290af" containerID="5205cbf648e73da307a6ff470401dcb1f181cfefa6a0f0d7e8d70f4c5578d5ad" exitCode=0 Jul 08 23:35:51.849787 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.849789 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" Jul 08 23:35:51.850017 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.849817 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" event={"ID":"a7b5476f-0128-4204-86ec-97d7396290af","Type":"ContainerDied","Data":"5205cbf648e73da307a6ff470401dcb1f181cfefa6a0f0d7e8d70f4c5578d5ad"} Jul 08 23:35:51.850017 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.849872 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7" event={"ID":"a7b5476f-0128-4204-86ec-97d7396290af","Type":"ContainerDied","Data":"5a93bfccf2c65d22bd9ebf5a78ed1a5c06d7aefc7380968291e6e3f3467d64b0"} Jul 08 23:35:51.850017 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.849893 2548 scope.go:117] "RemoveContainer" containerID="5205cbf648e73da307a6ff470401dcb1f181cfefa6a0f0d7e8d70f4c5578d5ad" Jul 08 23:35:51.859558 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.859541 2548 scope.go:117] "RemoveContainer" containerID="083330d65084ef44ecda0ef99cc2c4abd5fa570abe0d4b1caedae4d4b7ec3f43" Jul 08 23:35:51.868443 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.868422 2548 scope.go:117] "RemoveContainer" containerID="5205cbf648e73da307a6ff470401dcb1f181cfefa6a0f0d7e8d70f4c5578d5ad" Jul 08 23:35:51.868761 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:35:51.868724 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5205cbf648e73da307a6ff470401dcb1f181cfefa6a0f0d7e8d70f4c5578d5ad\": container with ID starting with 5205cbf648e73da307a6ff470401dcb1f181cfefa6a0f0d7e8d70f4c5578d5ad not found: ID does not exist" containerID="5205cbf648e73da307a6ff470401dcb1f181cfefa6a0f0d7e8d70f4c5578d5ad" Jul 08 23:35:51.868831 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.868771 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5205cbf648e73da307a6ff470401dcb1f181cfefa6a0f0d7e8d70f4c5578d5ad"} err="failed to get container status \"5205cbf648e73da307a6ff470401dcb1f181cfefa6a0f0d7e8d70f4c5578d5ad\": rpc error: code = NotFound desc = could not find container \"5205cbf648e73da307a6ff470401dcb1f181cfefa6a0f0d7e8d70f4c5578d5ad\": container with ID starting with 5205cbf648e73da307a6ff470401dcb1f181cfefa6a0f0d7e8d70f4c5578d5ad not found: ID does not exist" Jul 08 23:35:51.868831 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.868789 2548 scope.go:117] "RemoveContainer" containerID="083330d65084ef44ecda0ef99cc2c4abd5fa570abe0d4b1caedae4d4b7ec3f43" Jul 08 23:35:51.869061 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:35:51.869025 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"083330d65084ef44ecda0ef99cc2c4abd5fa570abe0d4b1caedae4d4b7ec3f43\": container with ID starting with 083330d65084ef44ecda0ef99cc2c4abd5fa570abe0d4b1caedae4d4b7ec3f43 not found: ID does not exist" containerID="083330d65084ef44ecda0ef99cc2c4abd5fa570abe0d4b1caedae4d4b7ec3f43" Jul 08 23:35:51.869163 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.869083 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"083330d65084ef44ecda0ef99cc2c4abd5fa570abe0d4b1caedae4d4b7ec3f43"} err="failed to get container status \"083330d65084ef44ecda0ef99cc2c4abd5fa570abe0d4b1caedae4d4b7ec3f43\": rpc error: code = NotFound desc = could not find container \"083330d65084ef44ecda0ef99cc2c4abd5fa570abe0d4b1caedae4d4b7ec3f43\": container with ID starting with 083330d65084ef44ecda0ef99cc2c4abd5fa570abe0d4b1caedae4d4b7ec3f43 not found: ID does not exist" Jul 08 23:35:51.871963 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.871938 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7"] Jul 08 23:35:51.875967 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:51.875945 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-e9d26a-openshift-default-56b666d485-bkcv7"] Jul 08 23:35:52.829635 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:52.829582 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7b5476f-0128-4204-86ec-97d7396290af" path="/var/lib/kubelet/pods/a7b5476f-0128-4204-86ec-97d7396290af/volumes" Jul 08 23:35:53.032807 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:53.032774 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:35:54.898824 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:54.898792 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb"] Jul 08 23:35:54.899274 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:54.899162 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7b5476f-0128-4204-86ec-97d7396290af" containerName="istio-proxy" Jul 08 23:35:54.899274 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:54.899174 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7b5476f-0128-4204-86ec-97d7396290af" containerName="istio-proxy" Jul 08 23:35:54.899274 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:54.899242 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7b5476f-0128-4204-86ec-97d7396290af" containerName="istio-proxy" Jul 08 23:35:54.899274 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:54.899251 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7b5476f-0128-4204-86ec-97d7396290af" containerName="istio-proxy" Jul 08 23:35:54.903430 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:54.903407 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:54.905861 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:54.905842 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-615023-openshift-default-dockercfg-xrn4s\"" Jul 08 23:35:54.915672 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:54.915643 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb"] Jul 08 23:35:55.067026 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.066992 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-token\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.067365 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.067341 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-credential-socket\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.067562 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.067531 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-envoy\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.067735 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.067718 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpjw6\" (UniqueName: \"kubernetes.io/projected/c3f23b2c-5001-494f-8205-6118f5149b1a-kube-api-access-kpjw6\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.067915 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.067889 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-workload-socket\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.068069 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.068052 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3f23b2c-5001-494f-8205-6118f5149b1a-istiod-ca-cert\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.068226 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.068211 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-data\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.068396 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.068371 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-podinfo\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.068571 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.068542 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-workload-certs\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.170053 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.169949 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-workload-socket\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.170724 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.170683 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3f23b2c-5001-494f-8205-6118f5149b1a-istiod-ca-cert\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.170844 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.170746 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-data\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.170844 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.170791 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-podinfo\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.170919 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.170843 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-workload-certs\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.170919 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.170851 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-workload-socket\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.170919 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.170907 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-token\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.171020 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.170960 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-credential-socket\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.171055 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.171023 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-envoy\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.171055 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.171049 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpjw6\" (UniqueName: \"kubernetes.io/projected/c3f23b2c-5001-494f-8205-6118f5149b1a-kube-api-access-kpjw6\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.171125 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.171087 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-data\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.171273 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.171252 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3f23b2c-5001-494f-8205-6118f5149b1a-istiod-ca-cert\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.171491 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.171465 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-credential-socket\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.171567 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.171540 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-workload-certs\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.173420 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.173402 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-envoy\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.173769 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.173740 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-podinfo\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.178866 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.178842 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-token\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.179311 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.179294 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpjw6\" (UniqueName: \"kubernetes.io/projected/c3f23b2c-5001-494f-8205-6118f5149b1a-kube-api-access-kpjw6\") pod \"e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.215633 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.215584 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:55.403516 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.403491 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb"] Jul 08 23:35:55.405519 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:35:55.405484 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3f23b2c_5001_494f_8205_6118f5149b1a.slice/crio-c61920cf7b8a9d165eb73e463446de164fdf7ded3aca76d5c578f4b8fef6845a WatchSource:0}: Error finding container c61920cf7b8a9d165eb73e463446de164fdf7ded3aca76d5c578f4b8fef6845a: Status 404 returned error can't find the container with id c61920cf7b8a9d165eb73e463446de164fdf7ded3aca76d5c578f4b8fef6845a Jul 08 23:35:55.407573 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.407543 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:35:55.407665 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.407627 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:35:55.407711 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.407667 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:35:55.882282 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.882246 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" event={"ID":"c3f23b2c-5001-494f-8205-6118f5149b1a","Type":"ContainerStarted","Data":"7db7c13df2904c39464b048ae5843611fc1da022ceee37e8a88d8dffa4d33c3e"} Jul 08 23:35:55.882499 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.882305 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" event={"ID":"c3f23b2c-5001-494f-8205-6118f5149b1a","Type":"ContainerStarted","Data":"c61920cf7b8a9d165eb73e463446de164fdf7ded3aca76d5c578f4b8fef6845a"} Jul 08 23:35:55.908619 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:55.908553 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" podStartSLOduration=1.908537662 podStartE2EDuration="1.908537662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:35:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:35:55.905386421 +0000 UTC m=+1977.665881470" watchObservedRunningTime="2026-07-08 23:35:55.908537662 +0000 UTC m=+1977.669032715" Jul 08 23:35:56.216665 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:56.216553 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:56.217832 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:56.217808 2548 patch_prober.go:28] interesting pod/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 08 23:35:56.217938 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:56.217878 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" podUID="c3f23b2c-5001-494f-8205-6118f5149b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 08 23:35:57.220683 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:57.220653 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:57.892174 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:57.892145 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:35:57.893128 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:35:57.893109 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:36:00.908756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:00.908673 2548 generic.go:356] "Generic (PLEG): container finished" podID="d9a81dd7-96fc-41dd-b82a-21b9ecf72700" containerID="a0e05b6eec0e7565ae782ce5ddd24f5094dd0624130524e4fe7b6b5850a81da2" exitCode=0 Jul 08 23:36:00.909200 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:00.908754 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" event={"ID":"d9a81dd7-96fc-41dd-b82a-21b9ecf72700","Type":"ContainerDied","Data":"a0e05b6eec0e7565ae782ce5ddd24f5094dd0624130524e4fe7b6b5850a81da2"} Jul 08 23:36:00.909200 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:00.909169 2548 scope.go:117] "RemoveContainer" containerID="a0e05b6eec0e7565ae782ce5ddd24f5094dd0624130524e4fe7b6b5850a81da2" Jul 08 23:36:00.910208 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:00.910182 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:36:00.910270 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:00.910249 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:36:00.910306 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:00.910283 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:36:01.917138 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:01.917104 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" event={"ID":"d9a81dd7-96fc-41dd-b82a-21b9ecf72700","Type":"ContainerStarted","Data":"eb112f2ef5976e9841247303d8c7a1463411cfafd2574d87556ad14ac0110434"} Jul 08 23:36:02.207332 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:02.207233 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:36:02.208488 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:02.208463 2548 patch_prober.go:28] interesting pod/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 08 23:36:02.208643 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:02.208513 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" podUID="d9a81dd7-96fc-41dd-b82a-21b9ecf72700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 08 23:36:03.207305 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:03.207266 2548 patch_prober.go:28] interesting pod/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 08 23:36:03.207752 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:03.207323 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" podUID="d9a81dd7-96fc-41dd-b82a-21b9ecf72700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 08 23:36:03.830283 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:03.830249 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:36:04.206981 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:04.206891 2548 patch_prober.go:28] interesting pod/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 08 23:36:04.206981 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:04.206941 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" podUID="d9a81dd7-96fc-41dd-b82a-21b9ecf72700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 08 23:36:05.206747 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:05.206700 2548 patch_prober.go:28] interesting pod/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 08 23:36:05.207136 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:05.206770 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" podUID="d9a81dd7-96fc-41dd-b82a-21b9ecf72700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 08 23:36:06.210685 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:06.210646 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:36:06.211640 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:06.211616 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:36:11.799466 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:11.799431 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:36:11.981489 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:11.981457 2548 generic.go:356] "Generic (PLEG): container finished" podID="c3f23b2c-5001-494f-8205-6118f5149b1a" containerID="7db7c13df2904c39464b048ae5843611fc1da022ceee37e8a88d8dffa4d33c3e" exitCode=0 Jul 08 23:36:11.981695 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:11.981515 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" event={"ID":"c3f23b2c-5001-494f-8205-6118f5149b1a","Type":"ContainerDied","Data":"7db7c13df2904c39464b048ae5843611fc1da022ceee37e8a88d8dffa4d33c3e"} Jul 08 23:36:11.981950 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:11.981935 2548 scope.go:117] "RemoveContainer" containerID="7db7c13df2904c39464b048ae5843611fc1da022ceee37e8a88d8dffa4d33c3e" Jul 08 23:36:11.983081 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:11.983050 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:36:11.983158 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:11.983136 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:36:11.983202 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:11.983193 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:36:12.892801 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:12.892743 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:36:12.989062 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:12.989030 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" event={"ID":"c3f23b2c-5001-494f-8205-6118f5149b1a","Type":"ContainerStarted","Data":"7dd7506938d75f9b426d3ccaeff4307ab1bf01f0ca1391986ddbc1dc1c5b6247"} Jul 08 23:36:13.216332 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:13.216227 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:36:13.217487 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:13.217458 2548 patch_prober.go:28] interesting pod/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 08 23:36:13.217625 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:13.217539 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" podUID="c3f23b2c-5001-494f-8205-6118f5149b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 08 23:36:14.216838 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:14.216797 2548 patch_prober.go:28] interesting pod/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 08 23:36:14.217250 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:14.216855 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" podUID="c3f23b2c-5001-494f-8205-6118f5149b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 08 23:36:15.216656 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:15.216618 2548 patch_prober.go:28] interesting pod/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 08 23:36:15.216877 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:15.216682 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" podUID="c3f23b2c-5001-494f-8205-6118f5149b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 08 23:36:16.215975 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:16.215939 2548 patch_prober.go:28] interesting pod/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 08 23:36:16.216157 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:16.215996 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" podUID="c3f23b2c-5001-494f-8205-6118f5149b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 08 23:36:16.478125 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:16.477147 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w"] Jul 08 23:36:16.478125 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:16.477412 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" podUID="d9a81dd7-96fc-41dd-b82a-21b9ecf72700" containerName="istio-proxy" containerID="cri-o://eb112f2ef5976e9841247303d8c7a1463411cfafd2574d87556ad14ac0110434" gracePeriod=30 Jul 08 23:36:17.220153 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:17.220121 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:36:17.220401 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:17.220376 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:36:17.221192 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:17.221168 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:36:21.213856 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.213816 2548 patch_prober.go:28] interesting pod/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 23:36:21.214244 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.213874 2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" podUID="d9a81dd7-96fc-41dd-b82a-21b9ecf72700" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 23:36:21.749069 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.749007 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:36:21.843486 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.843441 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-credential-socket\") pod \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " Jul 08 23:36:21.843685 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.843526 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istiod-ca-cert\") pod \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " Jul 08 23:36:21.843685 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.843609 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-envoy\") pod \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " Jul 08 23:36:21.843685 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.843640 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66kbz\" (UniqueName: \"kubernetes.io/projected/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-kube-api-access-66kbz\") pod \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " Jul 08 23:36:21.843872 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.843771 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-workload-certs\") pod \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " Jul 08 23:36:21.843872 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.843783 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d9a81dd7-96fc-41dd-b82a-21b9ecf72700" (UID: "d9a81dd7-96fc-41dd-b82a-21b9ecf72700"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:36:21.843872 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.843827 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-podinfo\") pod \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " Jul 08 23:36:21.843872 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.843851 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-token\") pod \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " Jul 08 23:36:21.844081 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.843878 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-data\") pod \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " Jul 08 23:36:21.844081 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.843921 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-workload-socket\") pod \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\" (UID: \"d9a81dd7-96fc-41dd-b82a-21b9ecf72700\") " Jul 08 23:36:21.844081 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.844052 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d9a81dd7-96fc-41dd-b82a-21b9ecf72700" (UID: "d9a81dd7-96fc-41dd-b82a-21b9ecf72700"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:36:21.844226 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.844081 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d9a81dd7-96fc-41dd-b82a-21b9ecf72700" (UID: "d9a81dd7-96fc-41dd-b82a-21b9ecf72700"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:36:21.844341 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.844321 2548 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-workload-certs\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:36:21.844409 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.844347 2548 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-credential-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:36:21.844409 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.844361 2548 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istiod-ca-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:36:21.844753 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.844725 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d9a81dd7-96fc-41dd-b82a-21b9ecf72700" (UID: "d9a81dd7-96fc-41dd-b82a-21b9ecf72700"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:36:21.844926 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.844905 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-data" (OuterVolumeSpecName: "istio-data") pod "d9a81dd7-96fc-41dd-b82a-21b9ecf72700" (UID: "d9a81dd7-96fc-41dd-b82a-21b9ecf72700"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:36:21.846396 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.846366 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d9a81dd7-96fc-41dd-b82a-21b9ecf72700" (UID: "d9a81dd7-96fc-41dd-b82a-21b9ecf72700"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:36:21.846570 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.846545 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-kube-api-access-66kbz" (OuterVolumeSpecName: "kube-api-access-66kbz") pod "d9a81dd7-96fc-41dd-b82a-21b9ecf72700" (UID: "d9a81dd7-96fc-41dd-b82a-21b9ecf72700"). InnerVolumeSpecName "kube-api-access-66kbz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:36:21.846679 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.846566 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d9a81dd7-96fc-41dd-b82a-21b9ecf72700" (UID: "d9a81dd7-96fc-41dd-b82a-21b9ecf72700"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 23:36:21.846813 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.846794 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-token" (OuterVolumeSpecName: "istio-token") pod "d9a81dd7-96fc-41dd-b82a-21b9ecf72700" (UID: "d9a81dd7-96fc-41dd-b82a-21b9ecf72700"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:36:21.945133 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.945096 2548 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-workload-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:36:21.945133 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.945129 2548 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-envoy\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:36:21.945133 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.945142 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66kbz\" (UniqueName: \"kubernetes.io/projected/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-kube-api-access-66kbz\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:36:21.945402 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.945154 2548 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-podinfo\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:36:21.945402 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.945166 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-token\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:36:21.945402 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:21.945178 2548 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a81dd7-96fc-41dd-b82a-21b9ecf72700-istio-data\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:36:22.038011 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:22.037920 2548 generic.go:356] "Generic (PLEG): container finished" podID="d9a81dd7-96fc-41dd-b82a-21b9ecf72700" containerID="eb112f2ef5976e9841247303d8c7a1463411cfafd2574d87556ad14ac0110434" exitCode=0 Jul 08 23:36:22.038011 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:22.037972 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" Jul 08 23:36:22.038234 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:22.038004 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" event={"ID":"d9a81dd7-96fc-41dd-b82a-21b9ecf72700","Type":"ContainerDied","Data":"eb112f2ef5976e9841247303d8c7a1463411cfafd2574d87556ad14ac0110434"} Jul 08 23:36:22.038234 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:22.038063 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w" event={"ID":"d9a81dd7-96fc-41dd-b82a-21b9ecf72700","Type":"ContainerDied","Data":"f93a27f4c59351e46be8e647e21b22f88bfae3f2903491839949c3966a700cfe"} Jul 08 23:36:22.038234 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:22.038087 2548 scope.go:117] "RemoveContainer" containerID="eb112f2ef5976e9841247303d8c7a1463411cfafd2574d87556ad14ac0110434" Jul 08 23:36:22.048769 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:22.048751 2548 scope.go:117] "RemoveContainer" containerID="a0e05b6eec0e7565ae782ce5ddd24f5094dd0624130524e4fe7b6b5850a81da2" Jul 08 23:36:22.057387 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:22.057299 2548 scope.go:117] "RemoveContainer" containerID="eb112f2ef5976e9841247303d8c7a1463411cfafd2574d87556ad14ac0110434" Jul 08 23:36:22.057848 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:36:22.057820 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb112f2ef5976e9841247303d8c7a1463411cfafd2574d87556ad14ac0110434\": container with ID starting with eb112f2ef5976e9841247303d8c7a1463411cfafd2574d87556ad14ac0110434 not found: ID does not exist" containerID="eb112f2ef5976e9841247303d8c7a1463411cfafd2574d87556ad14ac0110434" Jul 08 23:36:22.057968 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:22.057860 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb112f2ef5976e9841247303d8c7a1463411cfafd2574d87556ad14ac0110434"} err="failed to get container status \"eb112f2ef5976e9841247303d8c7a1463411cfafd2574d87556ad14ac0110434\": rpc error: code = NotFound desc = could not find container \"eb112f2ef5976e9841247303d8c7a1463411cfafd2574d87556ad14ac0110434\": container with ID starting with eb112f2ef5976e9841247303d8c7a1463411cfafd2574d87556ad14ac0110434 not found: ID does not exist" Jul 08 23:36:22.057968 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:22.057883 2548 scope.go:117] "RemoveContainer" containerID="a0e05b6eec0e7565ae782ce5ddd24f5094dd0624130524e4fe7b6b5850a81da2" Jul 08 23:36:22.058154 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:36:22.058133 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0e05b6eec0e7565ae782ce5ddd24f5094dd0624130524e4fe7b6b5850a81da2\": container with ID starting with a0e05b6eec0e7565ae782ce5ddd24f5094dd0624130524e4fe7b6b5850a81da2 not found: ID does not exist" containerID="a0e05b6eec0e7565ae782ce5ddd24f5094dd0624130524e4fe7b6b5850a81da2" Jul 08 23:36:22.058199 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:22.058161 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0e05b6eec0e7565ae782ce5ddd24f5094dd0624130524e4fe7b6b5850a81da2"} err="failed to get container status \"a0e05b6eec0e7565ae782ce5ddd24f5094dd0624130524e4fe7b6b5850a81da2\": rpc error: code = NotFound desc = could not find container \"a0e05b6eec0e7565ae782ce5ddd24f5094dd0624130524e4fe7b6b5850a81da2\": container with ID starting with a0e05b6eec0e7565ae782ce5ddd24f5094dd0624130524e4fe7b6b5850a81da2 not found: ID does not exist" Jul 08 23:36:22.060070 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:22.060052 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w"] Jul 08 23:36:22.063127 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:22.063107 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-0a9b6b-openshift-default-6c5988ccbc-tmz9w"] Jul 08 23:36:22.828967 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:22.828934 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9a81dd7-96fc-41dd-b82a-21b9ecf72700" path="/var/lib/kubelet/pods/d9a81dd7-96fc-41dd-b82a-21b9ecf72700/volumes" Jul 08 23:36:23.191400 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:23.191318 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb"] Jul 08 23:36:23.191546 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:23.191511 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" podUID="c3f23b2c-5001-494f-8205-6118f5149b1a" containerName="istio-proxy" containerID="cri-o://7dd7506938d75f9b426d3ccaeff4307ab1bf01f0ca1391986ddbc1dc1c5b6247" gracePeriod=30 Jul 08 23:36:24.010399 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.010363 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6"] Jul 08 23:36:24.010794 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.010751 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9a81dd7-96fc-41dd-b82a-21b9ecf72700" containerName="istio-proxy" Jul 08 23:36:24.010794 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.010765 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9a81dd7-96fc-41dd-b82a-21b9ecf72700" containerName="istio-proxy" Jul 08 23:36:24.010794 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.010776 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7b5476f-0128-4204-86ec-97d7396290af" containerName="istio-proxy" Jul 08 23:36:24.010794 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.010782 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7b5476f-0128-4204-86ec-97d7396290af" containerName="istio-proxy" Jul 08 23:36:24.010794 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.010789 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9a81dd7-96fc-41dd-b82a-21b9ecf72700" containerName="istio-proxy" Jul 08 23:36:24.010794 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.010794 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9a81dd7-96fc-41dd-b82a-21b9ecf72700" containerName="istio-proxy" Jul 08 23:36:24.010981 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.010871 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9a81dd7-96fc-41dd-b82a-21b9ecf72700" containerName="istio-proxy" Jul 08 23:36:24.010981 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.010880 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9a81dd7-96fc-41dd-b82a-21b9ecf72700" containerName="istio-proxy" Jul 08 23:36:24.015492 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.015472 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.018192 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.018176 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-45d83d-openshift-default-dockercfg-pxxqh\"" Jul 08 23:36:24.025468 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.025443 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6"] Jul 08 23:36:24.164088 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.164052 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-workload-socket\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.164297 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.164101 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-credential-socket\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.164297 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.164135 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-data\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.164297 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.164160 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-workload-certs\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.164297 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.164244 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-envoy\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.164297 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.164269 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-podinfo\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.164297 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.164290 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-token\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.164562 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.164312 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2hph\" (UniqueName: \"kubernetes.io/projected/adad1589-7f0b-4401-b2da-398f6b2ce572-kube-api-access-v2hph\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.164562 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.164374 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adad1589-7f0b-4401-b2da-398f6b2ce572-istiod-ca-cert\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.265313 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.265208 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-envoy\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.265313 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.265254 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-podinfo\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.265313 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.265295 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-token\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.265628 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.265318 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2hph\" (UniqueName: \"kubernetes.io/projected/adad1589-7f0b-4401-b2da-398f6b2ce572-kube-api-access-v2hph\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.265628 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.265357 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adad1589-7f0b-4401-b2da-398f6b2ce572-istiod-ca-cert\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.265628 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.265562 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-workload-socket\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.265804 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.265657 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-credential-socket\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.265804 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.265698 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-data\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.265804 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.265719 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-workload-certs\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.265968 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.265914 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-workload-socket\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.265968 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.265951 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-credential-socket\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.266083 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.266004 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-data\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.266140 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.266121 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-workload-certs\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.266400 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.266370 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adad1589-7f0b-4401-b2da-398f6b2ce572-istiod-ca-cert\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.267638 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.267620 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-envoy\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.267896 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.267876 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-podinfo\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.275539 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.275518 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-token\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.275725 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.275708 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2hph\" (UniqueName: \"kubernetes.io/projected/adad1589-7f0b-4401-b2da-398f6b2ce572-kube-api-access-v2hph\") pod \"e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.326837 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.326792 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:24.465121 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.465096 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6"] Jul 08 23:36:24.467396 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:36:24.467368 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadad1589_7f0b_4401_b2da_398f6b2ce572.slice/crio-44d081e4b20036d48613fde87f75fc786d5359fc045baf6b981d80877615289c WatchSource:0}: Error finding container 44d081e4b20036d48613fde87f75fc786d5359fc045baf6b981d80877615289c: Status 404 returned error can't find the container with id 44d081e4b20036d48613fde87f75fc786d5359fc045baf6b981d80877615289c Jul 08 23:36:24.469972 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.469935 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:36:24.470057 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.470022 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:36:24.470116 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:24.470100 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:36:25.056161 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:25.056124 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" event={"ID":"adad1589-7f0b-4401-b2da-398f6b2ce572","Type":"ContainerStarted","Data":"c97cefcae582ab6b655ab1f9676523a9e7fd228403aec4c849b49dc33de12cca"} Jul 08 23:36:25.056569 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:25.056187 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" event={"ID":"adad1589-7f0b-4401-b2da-398f6b2ce572","Type":"ContainerStarted","Data":"44d081e4b20036d48613fde87f75fc786d5359fc045baf6b981d80877615289c"} Jul 08 23:36:25.077898 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:25.077850 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" podStartSLOduration=2.077835133 podStartE2EDuration="2.077835133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:36:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:36:25.074796135 +0000 UTC m=+2006.835291203" watchObservedRunningTime="2026-07-08 23:36:25.077835133 +0000 UTC m=+2006.838330187" Jul 08 23:36:25.327479 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:25.327376 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:25.332396 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:25.332369 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:26.062668 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:26.062641 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:26.063704 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:26.063684 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:27.901062 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:27.901023 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:36:28.483343 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.483319 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:36:28.606068 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.606019 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-credential-socket\") pod \"c3f23b2c-5001-494f-8205-6118f5149b1a\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " Jul 08 23:36:28.606068 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.606080 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-workload-certs\") pod \"c3f23b2c-5001-494f-8205-6118f5149b1a\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " Jul 08 23:36:28.606294 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.606110 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpjw6\" (UniqueName: \"kubernetes.io/projected/c3f23b2c-5001-494f-8205-6118f5149b1a-kube-api-access-kpjw6\") pod \"c3f23b2c-5001-494f-8205-6118f5149b1a\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " Jul 08 23:36:28.606294 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.606224 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3f23b2c-5001-494f-8205-6118f5149b1a-istiod-ca-cert\") pod \"c3f23b2c-5001-494f-8205-6118f5149b1a\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " Jul 08 23:36:28.606376 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.606311 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-token\") pod \"c3f23b2c-5001-494f-8205-6118f5149b1a\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " Jul 08 23:36:28.606376 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.606320 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c3f23b2c-5001-494f-8205-6118f5149b1a" (UID: "c3f23b2c-5001-494f-8205-6118f5149b1a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:36:28.606376 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.606339 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c3f23b2c-5001-494f-8205-6118f5149b1a" (UID: "c3f23b2c-5001-494f-8205-6118f5149b1a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:36:28.606376 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.606350 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-workload-socket\") pod \"c3f23b2c-5001-494f-8205-6118f5149b1a\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " Jul 08 23:36:28.606577 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.606397 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-data\") pod \"c3f23b2c-5001-494f-8205-6118f5149b1a\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " Jul 08 23:36:28.606577 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.606424 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-podinfo\") pod \"c3f23b2c-5001-494f-8205-6118f5149b1a\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " Jul 08 23:36:28.606577 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.606455 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-envoy\") pod \"c3f23b2c-5001-494f-8205-6118f5149b1a\" (UID: \"c3f23b2c-5001-494f-8205-6118f5149b1a\") " Jul 08 23:36:28.606577 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.606557 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3f23b2c-5001-494f-8205-6118f5149b1a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c3f23b2c-5001-494f-8205-6118f5149b1a" (UID: "c3f23b2c-5001-494f-8205-6118f5149b1a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:36:28.606866 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.606660 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c3f23b2c-5001-494f-8205-6118f5149b1a" (UID: "c3f23b2c-5001-494f-8205-6118f5149b1a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:36:28.606866 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.606787 2548 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-workload-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:36:28.606866 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.606806 2548 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-credential-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:36:28.606866 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.606822 2548 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-workload-certs\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:36:28.606866 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.606840 2548 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3f23b2c-5001-494f-8205-6118f5149b1a-istiod-ca-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:36:28.607271 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.607249 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-data" (OuterVolumeSpecName: "istio-data") pod "c3f23b2c-5001-494f-8205-6118f5149b1a" (UID: "c3f23b2c-5001-494f-8205-6118f5149b1a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:36:28.608537 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.608514 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c3f23b2c-5001-494f-8205-6118f5149b1a" (UID: "c3f23b2c-5001-494f-8205-6118f5149b1a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 23:36:28.608744 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.608545 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-token" (OuterVolumeSpecName: "istio-token") pod "c3f23b2c-5001-494f-8205-6118f5149b1a" (UID: "c3f23b2c-5001-494f-8205-6118f5149b1a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:36:28.608816 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.608767 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3f23b2c-5001-494f-8205-6118f5149b1a-kube-api-access-kpjw6" (OuterVolumeSpecName: "kube-api-access-kpjw6") pod "c3f23b2c-5001-494f-8205-6118f5149b1a" (UID: "c3f23b2c-5001-494f-8205-6118f5149b1a"). InnerVolumeSpecName "kube-api-access-kpjw6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:36:28.608816 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.608810 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c3f23b2c-5001-494f-8205-6118f5149b1a" (UID: "c3f23b2c-5001-494f-8205-6118f5149b1a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:36:28.707614 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.707566 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpjw6\" (UniqueName: \"kubernetes.io/projected/c3f23b2c-5001-494f-8205-6118f5149b1a-kube-api-access-kpjw6\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:36:28.707614 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.707609 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-token\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:36:28.707614 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.707620 2548 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-data\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:36:28.707863 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.707628 2548 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-podinfo\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:36:28.707863 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:28.707636 2548 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3f23b2c-5001-494f-8205-6118f5149b1a-istio-envoy\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:36:29.084090 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:29.084054 2548 generic.go:356] "Generic (PLEG): container finished" podID="c3f23b2c-5001-494f-8205-6118f5149b1a" containerID="7dd7506938d75f9b426d3ccaeff4307ab1bf01f0ca1391986ddbc1dc1c5b6247" exitCode=0 Jul 08 23:36:29.084477 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:29.084118 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" Jul 08 23:36:29.084477 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:29.084144 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" event={"ID":"c3f23b2c-5001-494f-8205-6118f5149b1a","Type":"ContainerDied","Data":"7dd7506938d75f9b426d3ccaeff4307ab1bf01f0ca1391986ddbc1dc1c5b6247"} Jul 08 23:36:29.084477 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:29.084238 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb" event={"ID":"c3f23b2c-5001-494f-8205-6118f5149b1a","Type":"ContainerDied","Data":"c61920cf7b8a9d165eb73e463446de164fdf7ded3aca76d5c578f4b8fef6845a"} Jul 08 23:36:29.084477 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:29.084266 2548 scope.go:117] "RemoveContainer" containerID="7dd7506938d75f9b426d3ccaeff4307ab1bf01f0ca1391986ddbc1dc1c5b6247" Jul 08 23:36:29.092944 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:29.092911 2548 scope.go:117] "RemoveContainer" containerID="7db7c13df2904c39464b048ae5843611fc1da022ceee37e8a88d8dffa4d33c3e" Jul 08 23:36:29.101062 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:29.101028 2548 scope.go:117] "RemoveContainer" containerID="7dd7506938d75f9b426d3ccaeff4307ab1bf01f0ca1391986ddbc1dc1c5b6247" Jul 08 23:36:29.101335 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:36:29.101309 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dd7506938d75f9b426d3ccaeff4307ab1bf01f0ca1391986ddbc1dc1c5b6247\": container with ID starting with 7dd7506938d75f9b426d3ccaeff4307ab1bf01f0ca1391986ddbc1dc1c5b6247 not found: ID does not exist" containerID="7dd7506938d75f9b426d3ccaeff4307ab1bf01f0ca1391986ddbc1dc1c5b6247" Jul 08 23:36:29.101404 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:29.101348 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dd7506938d75f9b426d3ccaeff4307ab1bf01f0ca1391986ddbc1dc1c5b6247"} err="failed to get container status \"7dd7506938d75f9b426d3ccaeff4307ab1bf01f0ca1391986ddbc1dc1c5b6247\": rpc error: code = NotFound desc = could not find container \"7dd7506938d75f9b426d3ccaeff4307ab1bf01f0ca1391986ddbc1dc1c5b6247\": container with ID starting with 7dd7506938d75f9b426d3ccaeff4307ab1bf01f0ca1391986ddbc1dc1c5b6247 not found: ID does not exist" Jul 08 23:36:29.101404 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:29.101373 2548 scope.go:117] "RemoveContainer" containerID="7db7c13df2904c39464b048ae5843611fc1da022ceee37e8a88d8dffa4d33c3e" Jul 08 23:36:29.101729 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:36:29.101709 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7db7c13df2904c39464b048ae5843611fc1da022ceee37e8a88d8dffa4d33c3e\": container with ID starting with 7db7c13df2904c39464b048ae5843611fc1da022ceee37e8a88d8dffa4d33c3e not found: ID does not exist" containerID="7db7c13df2904c39464b048ae5843611fc1da022ceee37e8a88d8dffa4d33c3e" Jul 08 23:36:29.101784 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:29.101735 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7db7c13df2904c39464b048ae5843611fc1da022ceee37e8a88d8dffa4d33c3e"} err="failed to get container status \"7db7c13df2904c39464b048ae5843611fc1da022ceee37e8a88d8dffa4d33c3e\": rpc error: code = NotFound desc = could not find container \"7db7c13df2904c39464b048ae5843611fc1da022ceee37e8a88d8dffa4d33c3e\": container with ID starting with 7db7c13df2904c39464b048ae5843611fc1da022ceee37e8a88d8dffa4d33c3e not found: ID does not exist" Jul 08 23:36:29.103078 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:29.103060 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb"] Jul 08 23:36:29.108071 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:29.108047 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-615023-openshift-default-85b57b7cf9-zc7bb"] Jul 08 23:36:30.829446 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:30.829413 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3f23b2c-5001-494f-8205-6118f5149b1a" path="/var/lib/kubelet/pods/c3f23b2c-5001-494f-8205-6118f5149b1a/volumes" Jul 08 23:36:31.995116 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:31.995085 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh"] Jul 08 23:36:31.995616 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:31.995580 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3f23b2c-5001-494f-8205-6118f5149b1a" containerName="istio-proxy" Jul 08 23:36:31.995685 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:31.995622 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3f23b2c-5001-494f-8205-6118f5149b1a" containerName="istio-proxy" Jul 08 23:36:31.995762 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:31.995750 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3f23b2c-5001-494f-8205-6118f5149b1a" containerName="istio-proxy" Jul 08 23:36:31.995807 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:31.995765 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3f23b2c-5001-494f-8205-6118f5149b1a" containerName="istio-proxy" Jul 08 23:36:31.998797 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:31.998778 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.001293 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.001271 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-59ce67-openshift-default-dockercfg-bhcl2\"" Jul 08 23:36:32.009195 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.009168 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh"] Jul 08 23:36:32.141077 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.141040 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-token\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.141077 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.141078 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-data\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.141360 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.141110 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbp7p\" (UniqueName: \"kubernetes.io/projected/0969d585-d9fb-495e-9aed-d6de3c07ea11-kube-api-access-lbp7p\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.141360 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.141196 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-workload-socket\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.141360 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.141261 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-podinfo\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.141360 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.141318 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-workload-certs\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.141360 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.141341 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-envoy\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.141360 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.141363 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-credential-socket\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.141715 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.141380 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0969d585-d9fb-495e-9aed-d6de3c07ea11-istiod-ca-cert\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.242118 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.242076 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-workload-certs\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.242118 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.242114 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-envoy\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.242378 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.242133 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-credential-socket\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.242378 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.242321 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0969d585-d9fb-495e-9aed-d6de3c07ea11-istiod-ca-cert\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.242378 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.242356 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-token\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.242378 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.242376 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-data\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.242627 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.242421 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbp7p\" (UniqueName: \"kubernetes.io/projected/0969d585-d9fb-495e-9aed-d6de3c07ea11-kube-api-access-lbp7p\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.242627 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.242469 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-credential-socket\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.242627 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.242482 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-workload-socket\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.242627 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.242544 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-workload-certs\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.242627 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.242569 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-podinfo\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.242906 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.242816 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-data\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.243061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.243039 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-workload-socket\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.243129 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.243049 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0969d585-d9fb-495e-9aed-d6de3c07ea11-istiod-ca-cert\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.244532 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.244511 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-envoy\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.244826 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.244809 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-podinfo\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.250065 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.250002 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-token\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.250168 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.250153 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbp7p\" (UniqueName: \"kubernetes.io/projected/0969d585-d9fb-495e-9aed-d6de3c07ea11-kube-api-access-lbp7p\") pod \"e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.310097 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.310067 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:32.454517 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.454487 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh"] Jul 08 23:36:32.456031 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:36:32.456004 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0969d585_d9fb_495e_9aed_d6de3c07ea11.slice/crio-e4864588f18c2a118902a48e6fb2fb95bd284719e5a4dd110da29b57129adf3a WatchSource:0}: Error finding container e4864588f18c2a118902a48e6fb2fb95bd284719e5a4dd110da29b57129adf3a: Status 404 returned error can't find the container with id e4864588f18c2a118902a48e6fb2fb95bd284719e5a4dd110da29b57129adf3a Jul 08 23:36:32.458146 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.458110 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:36:32.458234 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.458174 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:36:32.458234 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:32.458215 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:36:33.110751 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:33.110710 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" event={"ID":"0969d585-d9fb-495e-9aed-d6de3c07ea11","Type":"ContainerStarted","Data":"9132a5cd9ce59e817f6b8e7582c6c8dba26a89f6b13e98834dfaa7550470b60e"} Jul 08 23:36:33.111146 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:33.110777 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" event={"ID":"0969d585-d9fb-495e-9aed-d6de3c07ea11","Type":"ContainerStarted","Data":"e4864588f18c2a118902a48e6fb2fb95bd284719e5a4dd110da29b57129adf3a"} Jul 08 23:36:33.132636 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:33.132571 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" podStartSLOduration=2.1325549 podStartE2EDuration="2.1325549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:36:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:36:33.129789815 +0000 UTC m=+2014.890284869" watchObservedRunningTime="2026-07-08 23:36:33.1325549 +0000 UTC m=+2014.893049954" Jul 08 23:36:33.311220 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:33.311183 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:33.316232 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:33.316209 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:34.115902 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:34.115867 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:34.116718 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:34.116698 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:42.103783 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:42.103735 2548 patch_prober.go:28] interesting pod/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 23:36:42.104222 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:42.103808 2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" podUID="adad1589-7f0b-4401-b2da-398f6b2ce572" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 23:36:42.895101 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:42.895021 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:36:43.168651 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:43.168537 2548 generic.go:356] "Generic (PLEG): container finished" podID="adad1589-7f0b-4401-b2da-398f6b2ce572" containerID="c97cefcae582ab6b655ab1f9676523a9e7fd228403aec4c849b49dc33de12cca" exitCode=0 Jul 08 23:36:43.169045 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:43.168644 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" event={"ID":"adad1589-7f0b-4401-b2da-398f6b2ce572","Type":"ContainerDied","Data":"c97cefcae582ab6b655ab1f9676523a9e7fd228403aec4c849b49dc33de12cca"} Jul 08 23:36:43.169154 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:43.169133 2548 scope.go:117] "RemoveContainer" containerID="c97cefcae582ab6b655ab1f9676523a9e7fd228403aec4c849b49dc33de12cca" Jul 08 23:36:43.170252 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:43.170223 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:36:43.170330 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:43.170290 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:36:43.170330 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:43.170323 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:36:44.198619 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:44.198215 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" event={"ID":"adad1589-7f0b-4401-b2da-398f6b2ce572","Type":"ContainerStarted","Data":"762c4fe1c4522693e2cc4c7ec3991dcc17c64ae66a5bdd51d0c2e4d59e4b27e3"} Jul 08 23:36:44.327246 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:44.327211 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:44.327441 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:44.327420 2548 patch_prober.go:28] interesting pod/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 08 23:36:44.327501 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:44.327481 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" podUID="adad1589-7f0b-4401-b2da-398f6b2ce572" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 08 23:36:45.327970 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:45.327931 2548 patch_prober.go:28] interesting pod/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 08 23:36:45.328369 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:45.327989 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" podUID="adad1589-7f0b-4401-b2da-398f6b2ce572" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 08 23:36:46.327152 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:46.327114 2548 patch_prober.go:28] interesting pod/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 08 23:36:46.327338 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:46.327174 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" podUID="adad1589-7f0b-4401-b2da-398f6b2ce572" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 08 23:36:47.327361 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:47.327328 2548 patch_prober.go:28] interesting pod/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 08 23:36:47.327780 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:47.327381 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" podUID="adad1589-7f0b-4401-b2da-398f6b2ce572" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 08 23:36:48.330467 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:48.330397 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:48.330846 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:48.330711 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:48.331465 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:48.331447 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:36:50.157786 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:50.157743 2548 patch_prober.go:28] interesting pod/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 23:36:50.158255 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:50.157812 2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" podUID="0969d585-d9fb-495e-9aed-d6de3c07ea11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 23:36:51.239927 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:51.239895 2548 generic.go:356] "Generic (PLEG): container finished" podID="0969d585-d9fb-495e-9aed-d6de3c07ea11" containerID="9132a5cd9ce59e817f6b8e7582c6c8dba26a89f6b13e98834dfaa7550470b60e" exitCode=0 Jul 08 23:36:51.240394 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:51.239957 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" event={"ID":"0969d585-d9fb-495e-9aed-d6de3c07ea11","Type":"ContainerDied","Data":"9132a5cd9ce59e817f6b8e7582c6c8dba26a89f6b13e98834dfaa7550470b60e"} Jul 08 23:36:51.240394 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:51.240305 2548 scope.go:117] "RemoveContainer" containerID="9132a5cd9ce59e817f6b8e7582c6c8dba26a89f6b13e98834dfaa7550470b60e" Jul 08 23:36:51.241278 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:51.241248 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:36:51.241339 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:51.241318 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:36:51.241386 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:51.241352 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 23:36:52.252105 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:52.252061 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" event={"ID":"0969d585-d9fb-495e-9aed-d6de3c07ea11","Type":"ContainerStarted","Data":"458ff4ce2c9025612b2fce0d2a5168335585d47e69e6679372d2bdc9caf840c9"} Jul 08 23:36:52.310667 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:52.310636 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:52.310867 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:52.310845 2548 patch_prober.go:28] interesting pod/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 08 23:36:52.310912 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:52.310892 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" podUID="0969d585-d9fb-495e-9aed-d6de3c07ea11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 08 23:36:53.310907 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:53.310877 2548 patch_prober.go:28] interesting pod/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 08 23:36:53.311291 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:53.310925 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" podUID="0969d585-d9fb-495e-9aed-d6de3c07ea11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 08 23:36:54.311440 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:54.311410 2548 patch_prober.go:28] interesting pod/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 08 23:36:54.311944 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:54.311467 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" podUID="0969d585-d9fb-495e-9aed-d6de3c07ea11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 08 23:36:55.310641 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:55.310585 2548 patch_prober.go:28] interesting pod/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 08 23:36:55.310804 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:55.310664 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" podUID="0969d585-d9fb-495e-9aed-d6de3c07ea11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 08 23:36:56.314703 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:56.314675 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:56.315146 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:56.315039 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:36:56.315691 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:36:56.315672 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:37:03.695706 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:37:03.695667 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:37:22.298543 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:37:22.298464 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:37:51.501057 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:37:51.501020 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:38:10.197627 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:38:10.197573 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:38:23.804969 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:38:23.804934 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:38:40.900183 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:38:40.900145 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:38:58.192346 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:38:58.192267 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:39:15.096395 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:15.096357 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:39:31.703010 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.702970 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp"] Jul 08 23:39:31.703497 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.703479 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3f23b2c-5001-494f-8205-6118f5149b1a" containerName="istio-proxy" Jul 08 23:39:31.703576 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.703500 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3f23b2c-5001-494f-8205-6118f5149b1a" containerName="istio-proxy" Jul 08 23:39:31.706672 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.706653 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:31.709254 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.709228 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-45d83d\"/\"kube-root-ca.crt\"" Jul 08 23:39:31.710352 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.710325 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-45d83d\"/\"default-dockercfg-lblj9\"" Jul 08 23:39:31.710465 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.710351 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-45d83d\"/\"openshift-service-ca.crt\"" Jul 08 23:39:31.710465 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.710371 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-45d83d\"/\"sub-test-model-45d83d-kserve-self-signed-certs\"" Jul 08 23:39:31.716586 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.716564 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp"] Jul 08 23:39:31.872020 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.871986 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fd78b58-a584-44c7-9895-d2291a83c7b5-tls-certs\") pod \"sub-test-model-45d83d-kserve-7fc4c94979-ft4gp\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:31.872222 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.872034 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-home\") pod \"sub-test-model-45d83d-kserve-7fc4c94979-ft4gp\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:31.872222 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.872118 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnp7z\" (UniqueName: \"kubernetes.io/projected/5fd78b58-a584-44c7-9895-d2291a83c7b5-kube-api-access-tnp7z\") pod \"sub-test-model-45d83d-kserve-7fc4c94979-ft4gp\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:31.872222 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.872166 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-tmp-dir\") pod \"sub-test-model-45d83d-kserve-7fc4c94979-ft4gp\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:31.872222 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.872196 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-dshm\") pod \"sub-test-model-45d83d-kserve-7fc4c94979-ft4gp\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:31.872385 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.872229 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-model-cache\") pod \"sub-test-model-45d83d-kserve-7fc4c94979-ft4gp\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:31.973067 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.972975 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fd78b58-a584-44c7-9895-d2291a83c7b5-tls-certs\") pod \"sub-test-model-45d83d-kserve-7fc4c94979-ft4gp\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:31.973067 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.973020 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-home\") pod \"sub-test-model-45d83d-kserve-7fc4c94979-ft4gp\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:31.973067 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.973053 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnp7z\" (UniqueName: \"kubernetes.io/projected/5fd78b58-a584-44c7-9895-d2291a83c7b5-kube-api-access-tnp7z\") pod \"sub-test-model-45d83d-kserve-7fc4c94979-ft4gp\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:31.973378 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.973204 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-tmp-dir\") pod \"sub-test-model-45d83d-kserve-7fc4c94979-ft4gp\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:31.973378 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.973242 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-dshm\") pod \"sub-test-model-45d83d-kserve-7fc4c94979-ft4gp\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:31.973378 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.973275 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-model-cache\") pod \"sub-test-model-45d83d-kserve-7fc4c94979-ft4gp\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:31.973565 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.973543 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-tmp-dir\") pod \"sub-test-model-45d83d-kserve-7fc4c94979-ft4gp\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:31.973668 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.973582 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-home\") pod \"sub-test-model-45d83d-kserve-7fc4c94979-ft4gp\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:31.973668 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.973650 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-model-cache\") pod \"sub-test-model-45d83d-kserve-7fc4c94979-ft4gp\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:31.975373 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.975355 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-dshm\") pod \"sub-test-model-45d83d-kserve-7fc4c94979-ft4gp\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:31.975799 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.975780 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fd78b58-a584-44c7-9895-d2291a83c7b5-tls-certs\") pod \"sub-test-model-45d83d-kserve-7fc4c94979-ft4gp\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:31.980510 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:31.980485 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnp7z\" (UniqueName: \"kubernetes.io/projected/5fd78b58-a584-44c7-9895-d2291a83c7b5-kube-api-access-tnp7z\") pod \"sub-test-model-45d83d-kserve-7fc4c94979-ft4gp\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:32.016558 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:32.016530 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:32.155713 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:32.155687 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp"] Jul 08 23:39:32.157439 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:39:32.157412 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5fd78b58_a584_44c7_9895_d2291a83c7b5.slice/crio-92dcf4529cce132f4082a74cc0fe1af0297a673dde4215801c70ab7389f2e6f5 WatchSource:0}: Error finding container 92dcf4529cce132f4082a74cc0fe1af0297a673dde4215801c70ab7389f2e6f5: Status 404 returned error can't find the container with id 92dcf4529cce132f4082a74cc0fe1af0297a673dde4215801c70ab7389f2e6f5 Jul 08 23:39:32.159825 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:32.159810 2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 23:39:32.293390 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:32.293352 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:39:33.102194 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:33.102158 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" event={"ID":"5fd78b58-a584-44c7-9895-d2291a83c7b5","Type":"ContainerStarted","Data":"92dcf4529cce132f4082a74cc0fe1af0297a673dde4215801c70ab7389f2e6f5"} Jul 08 23:39:34.109962 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:34.109919 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" event={"ID":"5fd78b58-a584-44c7-9895-d2291a83c7b5","Type":"ContainerStarted","Data":"21ca551d07f9a3e931bf8451f8ecae93a6b4d678d9cbf0057e60d88a56e53527"} Jul 08 23:39:34.127940 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:34.127895 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" podStartSLOduration=1.978979204 podStartE2EDuration="3.127880683s" totalImagesPullingTime="1.148901479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:39:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:39:34.125173557 +0000 UTC m=+2195.885668610" watchObservedRunningTime="2026-07-08 23:39:34.127880683 +0000 UTC m=+2195.888375736" Jul 08 23:39:42.017429 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:42.017391 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:42.017878 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:42.017470 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:42.029668 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:42.029645 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:42.170713 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:42.170682 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:39:47.793168 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:39:47.793134 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:40:09.693519 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:40:09.693476 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:40:36.195046 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:40:36.195013 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:40:50.140094 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:40:50.140060 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp"] Jul 08 23:40:55.799719 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:40:55.799682 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:41:05.160817 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.160758 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" podUID="5fd78b58-a584-44c7-9895-d2291a83c7b5" containerName="main" containerID="cri-o://21ca551d07f9a3e931bf8451f8ecae93a6b4d678d9cbf0057e60d88a56e53527" gracePeriod=45 Jul 08 23:41:05.418590 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.418518 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:41:05.509647 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.509619 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnp7z\" (UniqueName: \"kubernetes.io/projected/5fd78b58-a584-44c7-9895-d2291a83c7b5-kube-api-access-tnp7z\") pod \"5fd78b58-a584-44c7-9895-d2291a83c7b5\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " Jul 08 23:41:05.509821 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.509667 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fd78b58-a584-44c7-9895-d2291a83c7b5-tls-certs\") pod \"5fd78b58-a584-44c7-9895-d2291a83c7b5\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " Jul 08 23:41:05.509821 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.509724 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-model-cache\") pod \"5fd78b58-a584-44c7-9895-d2291a83c7b5\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " Jul 08 23:41:05.509821 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.509764 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-home\") pod \"5fd78b58-a584-44c7-9895-d2291a83c7b5\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " Jul 08 23:41:05.509975 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.509843 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-tmp-dir\") pod \"5fd78b58-a584-44c7-9895-d2291a83c7b5\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " Jul 08 23:41:05.509975 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.509887 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-dshm\") pod \"5fd78b58-a584-44c7-9895-d2291a83c7b5\" (UID: \"5fd78b58-a584-44c7-9895-d2291a83c7b5\") " Jul 08 23:41:05.510081 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.510029 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-model-cache" (OuterVolumeSpecName: "model-cache") pod "5fd78b58-a584-44c7-9895-d2291a83c7b5" (UID: "5fd78b58-a584-44c7-9895-d2291a83c7b5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:41:05.510139 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.510092 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-home" (OuterVolumeSpecName: "home") pod "5fd78b58-a584-44c7-9895-d2291a83c7b5" (UID: "5fd78b58-a584-44c7-9895-d2291a83c7b5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:41:05.510139 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.510118 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5fd78b58-a584-44c7-9895-d2291a83c7b5" (UID: "5fd78b58-a584-44c7-9895-d2291a83c7b5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:41:05.510287 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.510264 2548 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-model-cache\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:41:05.510287 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.510289 2548 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-home\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:41:05.510470 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.510306 2548 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-tmp-dir\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:41:05.511972 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.511942 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-dshm" (OuterVolumeSpecName: "dshm") pod "5fd78b58-a584-44c7-9895-d2291a83c7b5" (UID: "5fd78b58-a584-44c7-9895-d2291a83c7b5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:41:05.512071 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.512032 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fd78b58-a584-44c7-9895-d2291a83c7b5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5fd78b58-a584-44c7-9895-d2291a83c7b5" (UID: "5fd78b58-a584-44c7-9895-d2291a83c7b5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 23:41:05.512071 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.512056 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fd78b58-a584-44c7-9895-d2291a83c7b5-kube-api-access-tnp7z" (OuterVolumeSpecName: "kube-api-access-tnp7z") pod "5fd78b58-a584-44c7-9895-d2291a83c7b5" (UID: "5fd78b58-a584-44c7-9895-d2291a83c7b5"). InnerVolumeSpecName "kube-api-access-tnp7z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:41:05.597813 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.597780 2548 generic.go:356] "Generic (PLEG): container finished" podID="5fd78b58-a584-44c7-9895-d2291a83c7b5" containerID="21ca551d07f9a3e931bf8451f8ecae93a6b4d678d9cbf0057e60d88a56e53527" exitCode=0 Jul 08 23:41:05.597977 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.597830 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" Jul 08 23:41:05.597977 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.597831 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" event={"ID":"5fd78b58-a584-44c7-9895-d2291a83c7b5","Type":"ContainerDied","Data":"21ca551d07f9a3e931bf8451f8ecae93a6b4d678d9cbf0057e60d88a56e53527"} Jul 08 23:41:05.597977 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.597942 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp" event={"ID":"5fd78b58-a584-44c7-9895-d2291a83c7b5","Type":"ContainerDied","Data":"92dcf4529cce132f4082a74cc0fe1af0297a673dde4215801c70ab7389f2e6f5"} Jul 08 23:41:05.597977 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.597967 2548 scope.go:117] "RemoveContainer" containerID="21ca551d07f9a3e931bf8451f8ecae93a6b4d678d9cbf0057e60d88a56e53527" Jul 08 23:41:05.607215 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.607197 2548 scope.go:117] "RemoveContainer" containerID="21ca551d07f9a3e931bf8451f8ecae93a6b4d678d9cbf0057e60d88a56e53527" Jul 08 23:41:05.607462 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:41:05.607444 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21ca551d07f9a3e931bf8451f8ecae93a6b4d678d9cbf0057e60d88a56e53527\": container with ID starting with 21ca551d07f9a3e931bf8451f8ecae93a6b4d678d9cbf0057e60d88a56e53527 not found: ID does not exist" containerID="21ca551d07f9a3e931bf8451f8ecae93a6b4d678d9cbf0057e60d88a56e53527" Jul 08 23:41:05.607535 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.607467 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21ca551d07f9a3e931bf8451f8ecae93a6b4d678d9cbf0057e60d88a56e53527"} err="failed to get container status \"21ca551d07f9a3e931bf8451f8ecae93a6b4d678d9cbf0057e60d88a56e53527\": rpc error: code = NotFound desc = could not find container \"21ca551d07f9a3e931bf8451f8ecae93a6b4d678d9cbf0057e60d88a56e53527\": container with ID starting with 21ca551d07f9a3e931bf8451f8ecae93a6b4d678d9cbf0057e60d88a56e53527 not found: ID does not exist" Jul 08 23:41:05.611669 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.611645 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnp7z\" (UniqueName: \"kubernetes.io/projected/5fd78b58-a584-44c7-9895-d2291a83c7b5-kube-api-access-tnp7z\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:41:05.611741 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.611676 2548 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fd78b58-a584-44c7-9895-d2291a83c7b5-tls-certs\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:41:05.611741 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.611686 2548 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fd78b58-a584-44c7-9895-d2291a83c7b5-dshm\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:41:05.620150 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.620126 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp"] Jul 08 23:41:05.622183 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:05.622162 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-45d83d/sub-test-model-45d83d-kserve-7fc4c94979-ft4gp"] Jul 08 23:41:06.829061 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:06.829029 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fd78b58-a584-44c7-9895-d2291a83c7b5" path="/var/lib/kubelet/pods/5fd78b58-a584-44c7-9895-d2291a83c7b5/volumes" Jul 08 23:41:07.902522 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:07.902489 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx"] Jul 08 23:41:07.902903 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:07.902849 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fd78b58-a584-44c7-9895-d2291a83c7b5" containerName="main" Jul 08 23:41:07.902903 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:07.902861 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fd78b58-a584-44c7-9895-d2291a83c7b5" containerName="main" Jul 08 23:41:07.902983 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:07.902920 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fd78b58-a584-44c7-9895-d2291a83c7b5" containerName="main" Jul 08 23:41:07.907298 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:07.907279 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:07.910958 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:07.910905 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-45d83d\"/\"rate-test-model-45d83d-kserve-self-signed-certs\"" Jul 08 23:41:07.911089 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:07.910957 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-45d83d\"/\"openshift-service-ca.crt\"" Jul 08 23:41:07.911089 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:07.910982 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-45d83d\"/\"default-dockercfg-lblj9\"" Jul 08 23:41:07.911089 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:07.910959 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-45d83d\"/\"kube-root-ca.crt\"" Jul 08 23:41:07.914003 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:07.913980 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx"] Jul 08 23:41:08.035009 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.034978 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-tmp-dir\") pod \"rate-test-model-45d83d-kserve-65c977c8cf-ddhbx\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:08.035191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.035030 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-dshm\") pod \"rate-test-model-45d83d-kserve-65c977c8cf-ddhbx\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:08.035191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.035058 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-model-cache\") pod \"rate-test-model-45d83d-kserve-65c977c8cf-ddhbx\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:08.035191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.035075 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b2d46fc-291c-4daf-baf4-2a4e54205d12-tls-certs\") pod \"rate-test-model-45d83d-kserve-65c977c8cf-ddhbx\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:08.035191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.035125 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-home\") pod \"rate-test-model-45d83d-kserve-65c977c8cf-ddhbx\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:08.035191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.035166 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wddx5\" (UniqueName: \"kubernetes.io/projected/5b2d46fc-291c-4daf-baf4-2a4e54205d12-kube-api-access-wddx5\") pod \"rate-test-model-45d83d-kserve-65c977c8cf-ddhbx\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:08.136394 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.136351 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-model-cache\") pod \"rate-test-model-45d83d-kserve-65c977c8cf-ddhbx\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:08.136394 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.136397 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b2d46fc-291c-4daf-baf4-2a4e54205d12-tls-certs\") pod \"rate-test-model-45d83d-kserve-65c977c8cf-ddhbx\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:08.136674 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.136436 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-home\") pod \"rate-test-model-45d83d-kserve-65c977c8cf-ddhbx\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:08.136674 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.136466 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wddx5\" (UniqueName: \"kubernetes.io/projected/5b2d46fc-291c-4daf-baf4-2a4e54205d12-kube-api-access-wddx5\") pod \"rate-test-model-45d83d-kserve-65c977c8cf-ddhbx\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:08.136674 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.136530 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-tmp-dir\") pod \"rate-test-model-45d83d-kserve-65c977c8cf-ddhbx\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:08.136674 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.136587 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-dshm\") pod \"rate-test-model-45d83d-kserve-65c977c8cf-ddhbx\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:08.136905 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.136806 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-model-cache\") pod \"rate-test-model-45d83d-kserve-65c977c8cf-ddhbx\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:08.136905 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.136863 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-home\") pod \"rate-test-model-45d83d-kserve-65c977c8cf-ddhbx\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:08.136905 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.136887 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-tmp-dir\") pod \"rate-test-model-45d83d-kserve-65c977c8cf-ddhbx\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:08.138718 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.138697 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-dshm\") pod \"rate-test-model-45d83d-kserve-65c977c8cf-ddhbx\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:08.138983 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.138965 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b2d46fc-291c-4daf-baf4-2a4e54205d12-tls-certs\") pod \"rate-test-model-45d83d-kserve-65c977c8cf-ddhbx\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:08.144094 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.144062 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wddx5\" (UniqueName: \"kubernetes.io/projected/5b2d46fc-291c-4daf-baf4-2a4e54205d12-kube-api-access-wddx5\") pod \"rate-test-model-45d83d-kserve-65c977c8cf-ddhbx\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:08.218012 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.217922 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:08.353135 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.353102 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx"] Jul 08 23:41:08.354786 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:41:08.354761 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b2d46fc_291c_4daf_baf4_2a4e54205d12.slice/crio-6eadd93df19892a0941e352796934e7fa08557637d149c586635b0be64997b6c WatchSource:0}: Error finding container 6eadd93df19892a0941e352796934e7fa08557637d149c586635b0be64997b6c: Status 404 returned error can't find the container with id 6eadd93df19892a0941e352796934e7fa08557637d149c586635b0be64997b6c Jul 08 23:41:08.615212 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.615176 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" event={"ID":"5b2d46fc-291c-4daf-baf4-2a4e54205d12","Type":"ContainerStarted","Data":"b3471c8584e1138ebd3984871793608d64562c5f466ff7272a8afa93d22d251e"} Jul 08 23:41:08.615396 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.615223 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" event={"ID":"5b2d46fc-291c-4daf-baf4-2a4e54205d12","Type":"ContainerStarted","Data":"6eadd93df19892a0941e352796934e7fa08557637d149c586635b0be64997b6c"} Jul 08 23:41:08.633050 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:08.633002 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" podStartSLOduration=1.632986851 podStartE2EDuration="1.632986851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:41:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:41:08.630660647 +0000 UTC m=+2290.391155702" watchObservedRunningTime="2026-07-08 23:41:08.632986851 +0000 UTC m=+2290.393481910" Jul 08 23:41:12.002265 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:12.002231 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:41:18.218796 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:18.218756 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:18.219163 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:18.218813 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:18.231188 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:18.231161 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:18.678961 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:18.678928 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:41:39.311568 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.311533 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2"] Jul 08 23:41:39.314955 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.314932 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.319304 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.319283 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-59ce67\"/\"kube-root-ca.crt\"" Jul 08 23:41:39.319418 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.319282 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-59ce67\"/\"default-dockercfg-mw654\"" Jul 08 23:41:39.319418 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.319324 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-59ce67\"/\"openshift-service-ca.crt\"" Jul 08 23:41:39.319418 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.319284 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-59ce67\"/\"rate-test-model-59ce67-kserve-self-signed-certs\"" Jul 08 23:41:39.323934 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.323914 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2"] Jul 08 23:41:39.414164 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.414132 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-dshm\") pod \"rate-test-model-59ce67-kserve-667dc65c6c-jtbw2\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.414164 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.414170 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-tmp-dir\") pod \"rate-test-model-59ce67-kserve-667dc65c6c-jtbw2\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.414426 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.414188 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-model-cache\") pod \"rate-test-model-59ce67-kserve-667dc65c6c-jtbw2\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.414426 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.414308 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-home\") pod \"rate-test-model-59ce67-kserve-667dc65c6c-jtbw2\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.414426 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.414402 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-tls-certs\") pod \"rate-test-model-59ce67-kserve-667dc65c6c-jtbw2\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.414576 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.414447 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hpmp\" (UniqueName: \"kubernetes.io/projected/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-kube-api-access-9hpmp\") pod \"rate-test-model-59ce67-kserve-667dc65c6c-jtbw2\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.515675 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.515637 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-dshm\") pod \"rate-test-model-59ce67-kserve-667dc65c6c-jtbw2\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.515858 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.515683 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-tmp-dir\") pod \"rate-test-model-59ce67-kserve-667dc65c6c-jtbw2\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.515858 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.515776 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-model-cache\") pod \"rate-test-model-59ce67-kserve-667dc65c6c-jtbw2\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.515858 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.515828 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-home\") pod \"rate-test-model-59ce67-kserve-667dc65c6c-jtbw2\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.515980 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.515906 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-tls-certs\") pod \"rate-test-model-59ce67-kserve-667dc65c6c-jtbw2\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.515980 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.515957 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hpmp\" (UniqueName: \"kubernetes.io/projected/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-kube-api-access-9hpmp\") pod \"rate-test-model-59ce67-kserve-667dc65c6c-jtbw2\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.516171 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.516140 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-tmp-dir\") pod \"rate-test-model-59ce67-kserve-667dc65c6c-jtbw2\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.516235 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.516190 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-home\") pod \"rate-test-model-59ce67-kserve-667dc65c6c-jtbw2\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.516286 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.516244 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-model-cache\") pod \"rate-test-model-59ce67-kserve-667dc65c6c-jtbw2\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.517982 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.517955 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-dshm\") pod \"rate-test-model-59ce67-kserve-667dc65c6c-jtbw2\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.518272 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.518257 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-tls-certs\") pod \"rate-test-model-59ce67-kserve-667dc65c6c-jtbw2\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.524454 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.524436 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hpmp\" (UniqueName: \"kubernetes.io/projected/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-kube-api-access-9hpmp\") pod \"rate-test-model-59ce67-kserve-667dc65c6c-jtbw2\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.626254 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.626166 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:39.762427 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.762399 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2"] Jul 08 23:41:39.763986 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:41:39.763958 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69280ab5_f199_47bb_bd3e_c2eb9b6ce1ad.slice/crio-f4c36f8a8dc42fb5aa64af3758a27249d1320ba3604a5f79a241a0ef5f627bab WatchSource:0}: Error finding container f4c36f8a8dc42fb5aa64af3758a27249d1320ba3604a5f79a241a0ef5f627bab: Status 404 returned error can't find the container with id f4c36f8a8dc42fb5aa64af3758a27249d1320ba3604a5f79a241a0ef5f627bab Jul 08 23:41:39.776934 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:39.776910 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" event={"ID":"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad","Type":"ContainerStarted","Data":"f4c36f8a8dc42fb5aa64af3758a27249d1320ba3604a5f79a241a0ef5f627bab"} Jul 08 23:41:40.783364 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:40.783325 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" event={"ID":"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad","Type":"ContainerStarted","Data":"18f3dcc34662b3372b2395211404da0ef444b4ec2ff7c5e339d3c5095f8f0bb6"} Jul 08 23:41:40.800526 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:40.800473 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" podStartSLOduration=1.8004588990000001 podStartE2EDuration="1.800458899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:41:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:41:40.799591187 +0000 UTC m=+2322.560086243" watchObservedRunningTime="2026-07-08 23:41:40.800458899 +0000 UTC m=+2322.560953943" Jul 08 23:41:43.801796 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:43.801763 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:41:49.627078 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:49.627032 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:49.627488 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:49.627089 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:49.639531 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:49.639500 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:41:49.844456 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:41:49.844421 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:42:09.237401 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:09.237362 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:42:24.798041 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:24.798006 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:42:30.339060 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:30.339028 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx"] Jul 08 23:42:31.211057 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:31.211026 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2"] Jul 08 23:42:43.104654 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:43.104617 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:42:45.358787 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.358719 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" podUID="5b2d46fc-291c-4daf-baf4-2a4e54205d12" containerName="main" containerID="cri-o://b3471c8584e1138ebd3984871793608d64562c5f466ff7272a8afa93d22d251e" gracePeriod=45 Jul 08 23:42:45.606292 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.606268 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:42:45.722150 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.722054 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-model-cache\") pod \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " Jul 08 23:42:45.722150 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.722126 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-home\") pod \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " Jul 08 23:42:45.722396 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.722181 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-tmp-dir\") pod \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " Jul 08 23:42:45.722396 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.722215 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wddx5\" (UniqueName: \"kubernetes.io/projected/5b2d46fc-291c-4daf-baf4-2a4e54205d12-kube-api-access-wddx5\") pod \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " Jul 08 23:42:45.722396 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.722236 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-dshm\") pod \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " Jul 08 23:42:45.722396 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.722255 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b2d46fc-291c-4daf-baf4-2a4e54205d12-tls-certs\") pod \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\" (UID: \"5b2d46fc-291c-4daf-baf4-2a4e54205d12\") " Jul 08 23:42:45.722396 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.722350 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-model-cache" (OuterVolumeSpecName: "model-cache") pod "5b2d46fc-291c-4daf-baf4-2a4e54205d12" (UID: "5b2d46fc-291c-4daf-baf4-2a4e54205d12"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:42:45.722698 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.722397 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-home" (OuterVolumeSpecName: "home") pod "5b2d46fc-291c-4daf-baf4-2a4e54205d12" (UID: "5b2d46fc-291c-4daf-baf4-2a4e54205d12"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:42:45.722698 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.722554 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5b2d46fc-291c-4daf-baf4-2a4e54205d12" (UID: "5b2d46fc-291c-4daf-baf4-2a4e54205d12"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:42:45.722698 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.722586 2548 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-model-cache\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:42:45.722698 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.722634 2548 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-home\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:42:45.724356 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.724333 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-dshm" (OuterVolumeSpecName: "dshm") pod "5b2d46fc-291c-4daf-baf4-2a4e54205d12" (UID: "5b2d46fc-291c-4daf-baf4-2a4e54205d12"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:42:45.724440 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.724382 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b2d46fc-291c-4daf-baf4-2a4e54205d12-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5b2d46fc-291c-4daf-baf4-2a4e54205d12" (UID: "5b2d46fc-291c-4daf-baf4-2a4e54205d12"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 23:42:45.724476 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.724437 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b2d46fc-291c-4daf-baf4-2a4e54205d12-kube-api-access-wddx5" (OuterVolumeSpecName: "kube-api-access-wddx5") pod "5b2d46fc-291c-4daf-baf4-2a4e54205d12" (UID: "5b2d46fc-291c-4daf-baf4-2a4e54205d12"). InnerVolumeSpecName "kube-api-access-wddx5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:42:45.823499 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.823467 2548 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-tmp-dir\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:42:45.823499 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.823494 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wddx5\" (UniqueName: \"kubernetes.io/projected/5b2d46fc-291c-4daf-baf4-2a4e54205d12-kube-api-access-wddx5\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:42:45.823499 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.823504 2548 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b2d46fc-291c-4daf-baf4-2a4e54205d12-dshm\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:42:45.823765 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:45.823513 2548 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b2d46fc-291c-4daf-baf4-2a4e54205d12-tls-certs\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:42:46.138087 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.138052 2548 generic.go:356] "Generic (PLEG): container finished" podID="5b2d46fc-291c-4daf-baf4-2a4e54205d12" containerID="b3471c8584e1138ebd3984871793608d64562c5f466ff7272a8afa93d22d251e" exitCode=0 Jul 08 23:42:46.138258 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.138102 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" Jul 08 23:42:46.138258 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.138136 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" event={"ID":"5b2d46fc-291c-4daf-baf4-2a4e54205d12","Type":"ContainerDied","Data":"b3471c8584e1138ebd3984871793608d64562c5f466ff7272a8afa93d22d251e"} Jul 08 23:42:46.138258 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.138177 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx" event={"ID":"5b2d46fc-291c-4daf-baf4-2a4e54205d12","Type":"ContainerDied","Data":"6eadd93df19892a0941e352796934e7fa08557637d149c586635b0be64997b6c"} Jul 08 23:42:46.138258 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.138195 2548 scope.go:117] "RemoveContainer" containerID="b3471c8584e1138ebd3984871793608d64562c5f466ff7272a8afa93d22d251e" Jul 08 23:42:46.147569 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.147545 2548 scope.go:117] "RemoveContainer" containerID="b3471c8584e1138ebd3984871793608d64562c5f466ff7272a8afa93d22d251e" Jul 08 23:42:46.147850 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:42:46.147834 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3471c8584e1138ebd3984871793608d64562c5f466ff7272a8afa93d22d251e\": container with ID starting with b3471c8584e1138ebd3984871793608d64562c5f466ff7272a8afa93d22d251e not found: ID does not exist" containerID="b3471c8584e1138ebd3984871793608d64562c5f466ff7272a8afa93d22d251e" Jul 08 23:42:46.147912 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.147862 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3471c8584e1138ebd3984871793608d64562c5f466ff7272a8afa93d22d251e"} err="failed to get container status \"b3471c8584e1138ebd3984871793608d64562c5f466ff7272a8afa93d22d251e\": rpc error: code = NotFound desc = could not find container \"b3471c8584e1138ebd3984871793608d64562c5f466ff7272a8afa93d22d251e\": container with ID starting with b3471c8584e1138ebd3984871793608d64562c5f466ff7272a8afa93d22d251e not found: ID does not exist" Jul 08 23:42:46.161199 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.161173 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx"] Jul 08 23:42:46.164069 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.164050 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-45d83d/rate-test-model-45d83d-kserve-65c977c8cf-ddhbx"] Jul 08 23:42:46.231232 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.231195 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" podUID="69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad" containerName="main" containerID="cri-o://18f3dcc34662b3372b2395211404da0ef444b4ec2ff7c5e339d3c5095f8f0bb6" gracePeriod=45 Jul 08 23:42:46.477921 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.477894 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:42:46.629793 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.629757 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-model-cache\") pod \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " Jul 08 23:42:46.629961 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.629826 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-home\") pod \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " Jul 08 23:42:46.629961 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.629869 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-tmp-dir\") pod \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " Jul 08 23:42:46.629961 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.629889 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hpmp\" (UniqueName: \"kubernetes.io/projected/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-kube-api-access-9hpmp\") pod \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " Jul 08 23:42:46.629961 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.629924 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-dshm\") pod \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " Jul 08 23:42:46.629961 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.629944 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-tls-certs\") pod \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\" (UID: \"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad\") " Jul 08 23:42:46.630279 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.630053 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-model-cache" (OuterVolumeSpecName: "model-cache") pod "69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad" (UID: "69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:42:46.630279 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.630124 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad" (UID: "69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:42:46.630279 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.630170 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-home" (OuterVolumeSpecName: "home") pod "69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad" (UID: "69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:42:46.630279 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.630258 2548 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-model-cache\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:42:46.630279 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.630276 2548 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-home\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:42:46.630494 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.630288 2548 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-tmp-dir\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:42:46.632196 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.632174 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-kube-api-access-9hpmp" (OuterVolumeSpecName: "kube-api-access-9hpmp") pod "69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad" (UID: "69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad"). InnerVolumeSpecName "kube-api-access-9hpmp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:42:46.632288 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.632239 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-dshm" (OuterVolumeSpecName: "dshm") pod "69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad" (UID: "69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:42:46.632288 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.632242 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad" (UID: "69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 23:42:46.730986 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.730961 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hpmp\" (UniqueName: \"kubernetes.io/projected/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-kube-api-access-9hpmp\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:42:46.730986 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.730987 2548 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-dshm\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:42:46.731159 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.730998 2548 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad-tls-certs\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:42:46.830204 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:46.830165 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b2d46fc-291c-4daf-baf4-2a4e54205d12" path="/var/lib/kubelet/pods/5b2d46fc-291c-4daf-baf4-2a4e54205d12/volumes" Jul 08 23:42:47.146535 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:47.146445 2548 generic.go:356] "Generic (PLEG): container finished" podID="69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad" containerID="18f3dcc34662b3372b2395211404da0ef444b4ec2ff7c5e339d3c5095f8f0bb6" exitCode=0 Jul 08 23:42:47.146535 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:47.146514 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" Jul 08 23:42:47.146756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:47.146522 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" event={"ID":"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad","Type":"ContainerDied","Data":"18f3dcc34662b3372b2395211404da0ef444b4ec2ff7c5e339d3c5095f8f0bb6"} Jul 08 23:42:47.146756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:47.146567 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2" event={"ID":"69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad","Type":"ContainerDied","Data":"f4c36f8a8dc42fb5aa64af3758a27249d1320ba3604a5f79a241a0ef5f627bab"} Jul 08 23:42:47.146756 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:47.146583 2548 scope.go:117] "RemoveContainer" containerID="18f3dcc34662b3372b2395211404da0ef444b4ec2ff7c5e339d3c5095f8f0bb6" Jul 08 23:42:47.157315 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:47.157288 2548 scope.go:117] "RemoveContainer" containerID="18f3dcc34662b3372b2395211404da0ef444b4ec2ff7c5e339d3c5095f8f0bb6" Jul 08 23:42:47.158271 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:42:47.158248 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18f3dcc34662b3372b2395211404da0ef444b4ec2ff7c5e339d3c5095f8f0bb6\": container with ID starting with 18f3dcc34662b3372b2395211404da0ef444b4ec2ff7c5e339d3c5095f8f0bb6 not found: ID does not exist" containerID="18f3dcc34662b3372b2395211404da0ef444b4ec2ff7c5e339d3c5095f8f0bb6" Jul 08 23:42:47.158385 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:47.158310 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18f3dcc34662b3372b2395211404da0ef444b4ec2ff7c5e339d3c5095f8f0bb6"} err="failed to get container status \"18f3dcc34662b3372b2395211404da0ef444b4ec2ff7c5e339d3c5095f8f0bb6\": rpc error: code = NotFound desc = could not find container \"18f3dcc34662b3372b2395211404da0ef444b4ec2ff7c5e339d3c5095f8f0bb6\": container with ID starting with 18f3dcc34662b3372b2395211404da0ef444b4ec2ff7c5e339d3c5095f8f0bb6 not found: ID does not exist" Jul 08 23:42:47.171284 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:47.171249 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2"] Jul 08 23:42:47.173466 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:47.173434 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-59ce67/rate-test-model-59ce67-kserve-667dc65c6c-jtbw2"] Jul 08 23:42:48.828793 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:48.828751 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad" path="/var/lib/kubelet/pods/69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad/volumes" Jul 08 23:42:58.388842 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:58.388810 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/test-curl-7671-140539381375456"] Jul 08 23:42:58.389236 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:58.389160 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b2d46fc-291c-4daf-baf4-2a4e54205d12" containerName="main" Jul 08 23:42:58.389236 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:58.389174 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b2d46fc-291c-4daf-baf4-2a4e54205d12" containerName="main" Jul 08 23:42:58.389236 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:58.389187 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad" containerName="main" Jul 08 23:42:58.389236 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:58.389193 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad" containerName="main" Jul 08 23:42:58.389358 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:58.389245 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="69280ab5-f199-47bb-bd3e-c2eb9b6ce1ad" containerName="main" Jul 08 23:42:58.389358 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:58.389252 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b2d46fc-291c-4daf-baf4-2a4e54205d12" containerName="main" Jul 08 23:42:58.393370 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:58.393351 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/test-curl-7671-140539381375456" Jul 08 23:42:58.396379 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:58.396353 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 23:42:58.397214 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:58.397189 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 23:42:58.397344 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:58.397198 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-47sht\"" Jul 08 23:42:58.399404 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:58.399379 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/test-curl-7671-140539381375456"] Jul 08 23:42:58.437320 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:58.437295 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gfvr\" (UniqueName: \"kubernetes.io/projected/ab66be46-5c0d-4475-a909-60af39531ce8-kube-api-access-9gfvr\") pod \"test-curl-7671-140539381375456\" (UID: \"ab66be46-5c0d-4475-a909-60af39531ce8\") " pod="odh-ai-gateway-infra/test-curl-7671-140539381375456" Jul 08 23:42:58.538270 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:58.538241 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gfvr\" (UniqueName: \"kubernetes.io/projected/ab66be46-5c0d-4475-a909-60af39531ce8-kube-api-access-9gfvr\") pod \"test-curl-7671-140539381375456\" (UID: \"ab66be46-5c0d-4475-a909-60af39531ce8\") " pod="odh-ai-gateway-infra/test-curl-7671-140539381375456" Jul 08 23:42:58.547066 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:58.547042 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gfvr\" (UniqueName: \"kubernetes.io/projected/ab66be46-5c0d-4475-a909-60af39531ce8-kube-api-access-9gfvr\") pod \"test-curl-7671-140539381375456\" (UID: \"ab66be46-5c0d-4475-a909-60af39531ce8\") " pod="odh-ai-gateway-infra/test-curl-7671-140539381375456" Jul 08 23:42:58.706942 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:58.706870 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-47sht\"" Jul 08 23:42:58.715032 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:58.715010 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/test-curl-7671-140539381375456" Jul 08 23:42:58.848460 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:58.848360 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/test-curl-7671-140539381375456"] Jul 08 23:42:58.851372 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:42:58.851348 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab66be46_5c0d_4475_a909_60af39531ce8.slice/crio-0b7e2138b8f9d0e30dc90ba31281f4db771561399167f62ab867c1af1407c576 WatchSource:0}: Error finding container 0b7e2138b8f9d0e30dc90ba31281f4db771561399167f62ab867c1af1407c576: Status 404 returned error can't find the container with id 0b7e2138b8f9d0e30dc90ba31281f4db771561399167f62ab867c1af1407c576 Jul 08 23:42:59.214142 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:42:59.214103 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/test-curl-7671-140539381375456" event={"ID":"ab66be46-5c0d-4475-a909-60af39531ce8","Type":"ContainerStarted","Data":"0b7e2138b8f9d0e30dc90ba31281f4db771561399167f62ab867c1af1407c576"} Jul 08 23:43:00.220749 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:00.220716 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/test-curl-7671-140539381375456" event={"ID":"ab66be46-5c0d-4475-a909-60af39531ce8","Type":"ContainerStarted","Data":"722c080d4d85ff64034aa20c988d0137b1d3921fce337c17a3362e639013422c"} Jul 08 23:43:00.235817 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:00.235771 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/test-curl-7671-140539381375456" podStartSLOduration=1.547781685 podStartE2EDuration="2.235755981s" totalImagesPullingTime="687.974296ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:42:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:43:00.235410852 +0000 UTC m=+2401.995905906" watchObservedRunningTime="2026-07-08 23:43:00.235755981 +0000 UTC m=+2401.996251035" Jul 08 23:43:10.273274 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:10.273239 2548 generic.go:356] "Generic (PLEG): container finished" podID="ab66be46-5c0d-4475-a909-60af39531ce8" containerID="722c080d4d85ff64034aa20c988d0137b1d3921fce337c17a3362e639013422c" exitCode=28 Jul 08 23:43:10.273712 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:10.273312 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/test-curl-7671-140539381375456" event={"ID":"ab66be46-5c0d-4475-a909-60af39531ce8","Type":"ContainerDied","Data":"722c080d4d85ff64034aa20c988d0137b1d3921fce337c17a3362e639013422c"} Jul 08 23:43:11.414341 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:11.414314 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/test-curl-7671-140539381375456" Jul 08 23:43:11.452191 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:11.452153 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/test-curl-7671-140539381375456"] Jul 08 23:43:11.454207 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:11.454186 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/test-curl-7671-140539381375456"] Jul 08 23:43:11.457836 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:11.457818 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gfvr\" (UniqueName: \"kubernetes.io/projected/ab66be46-5c0d-4475-a909-60af39531ce8-kube-api-access-9gfvr\") pod \"ab66be46-5c0d-4475-a909-60af39531ce8\" (UID: \"ab66be46-5c0d-4475-a909-60af39531ce8\") " Jul 08 23:43:11.459906 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:11.459872 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab66be46-5c0d-4475-a909-60af39531ce8-kube-api-access-9gfvr" (OuterVolumeSpecName: "kube-api-access-9gfvr") pod "ab66be46-5c0d-4475-a909-60af39531ce8" (UID: "ab66be46-5c0d-4475-a909-60af39531ce8"). InnerVolumeSpecName "kube-api-access-9gfvr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:43:11.558734 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:11.558644 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gfvr\" (UniqueName: \"kubernetes.io/projected/ab66be46-5c0d-4475-a909-60af39531ce8-kube-api-access-9gfvr\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:43:12.284194 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:12.284169 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/test-curl-7671-140539381375456" Jul 08 23:43:12.284375 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:12.284177 2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b7e2138b8f9d0e30dc90ba31281f4db771561399167f62ab867c1af1407c576" Jul 08 23:43:12.296445 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:43:12.296408 2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7671-140539381375456\" is forbidden: User \"system:node:ip-10-0-135-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-135-106.ec2.internal' and this object" podUID="ab66be46-5c0d-4475-a909-60af39531ce8" pod="odh-ai-gateway-infra/test-curl-7671-140539381375456" Jul 08 23:43:12.829560 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:12.829528 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab66be46-5c0d-4475-a909-60af39531ce8" path="/var/lib/kubelet/pods/ab66be46-5c0d-4475-a909-60af39531ce8/volumes" Jul 08 23:43:15.900892 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:15.900855 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:43:18.472995 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:18.472959 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6"] Jul 08 23:43:18.473460 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:18.473167 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" podUID="adad1589-7f0b-4401-b2da-398f6b2ce572" containerName="istio-proxy" containerID="cri-o://762c4fe1c4522693e2cc4c7ec3991dcc17c64ae66a5bdd51d0c2e4d59e4b27e3" gracePeriod=30 Jul 08 23:43:23.724180 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.724157 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:43:23.868320 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.868236 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-workload-certs\") pod \"adad1589-7f0b-4401-b2da-398f6b2ce572\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " Jul 08 23:43:23.868320 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.868287 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-workload-socket\") pod \"adad1589-7f0b-4401-b2da-398f6b2ce572\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " Jul 08 23:43:23.868562 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.868325 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adad1589-7f0b-4401-b2da-398f6b2ce572-istiod-ca-cert\") pod \"adad1589-7f0b-4401-b2da-398f6b2ce572\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " Jul 08 23:43:23.868562 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.868363 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-token\") pod \"adad1589-7f0b-4401-b2da-398f6b2ce572\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " Jul 08 23:43:23.868562 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.868415 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-podinfo\") pod \"adad1589-7f0b-4401-b2da-398f6b2ce572\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " Jul 08 23:43:23.868562 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.868545 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-data\") pod \"adad1589-7f0b-4401-b2da-398f6b2ce572\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " Jul 08 23:43:23.868810 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.868670 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2hph\" (UniqueName: \"kubernetes.io/projected/adad1589-7f0b-4401-b2da-398f6b2ce572-kube-api-access-v2hph\") pod \"adad1589-7f0b-4401-b2da-398f6b2ce572\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " Jul 08 23:43:23.868810 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.868551 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "adad1589-7f0b-4401-b2da-398f6b2ce572" (UID: "adad1589-7f0b-4401-b2da-398f6b2ce572"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:43:23.868810 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.868639 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "adad1589-7f0b-4401-b2da-398f6b2ce572" (UID: "adad1589-7f0b-4401-b2da-398f6b2ce572"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:43:23.868810 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.868696 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-credential-socket\") pod \"adad1589-7f0b-4401-b2da-398f6b2ce572\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " Jul 08 23:43:23.868810 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.868773 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-envoy\") pod \"adad1589-7f0b-4401-b2da-398f6b2ce572\" (UID: \"adad1589-7f0b-4401-b2da-398f6b2ce572\") " Jul 08 23:43:23.869073 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.868834 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adad1589-7f0b-4401-b2da-398f6b2ce572-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "adad1589-7f0b-4401-b2da-398f6b2ce572" (UID: "adad1589-7f0b-4401-b2da-398f6b2ce572"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:43:23.869073 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.868897 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "adad1589-7f0b-4401-b2da-398f6b2ce572" (UID: "adad1589-7f0b-4401-b2da-398f6b2ce572"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:43:23.869182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.869078 2548 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-workload-certs\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:43:23.869182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.869097 2548 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-workload-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:43:23.869182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.869112 2548 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adad1589-7f0b-4401-b2da-398f6b2ce572-istiod-ca-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:43:23.869182 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.869127 2548 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-credential-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:43:23.869625 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.869580 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-data" (OuterVolumeSpecName: "istio-data") pod "adad1589-7f0b-4401-b2da-398f6b2ce572" (UID: "adad1589-7f0b-4401-b2da-398f6b2ce572"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:43:23.870759 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.870731 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "adad1589-7f0b-4401-b2da-398f6b2ce572" (UID: "adad1589-7f0b-4401-b2da-398f6b2ce572"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:43:23.870759 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.870744 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-token" (OuterVolumeSpecName: "istio-token") pod "adad1589-7f0b-4401-b2da-398f6b2ce572" (UID: "adad1589-7f0b-4401-b2da-398f6b2ce572"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:43:23.870891 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.870741 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "adad1589-7f0b-4401-b2da-398f6b2ce572" (UID: "adad1589-7f0b-4401-b2da-398f6b2ce572"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 23:43:23.870891 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.870767 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adad1589-7f0b-4401-b2da-398f6b2ce572-kube-api-access-v2hph" (OuterVolumeSpecName: "kube-api-access-v2hph") pod "adad1589-7f0b-4401-b2da-398f6b2ce572" (UID: "adad1589-7f0b-4401-b2da-398f6b2ce572"). InnerVolumeSpecName "kube-api-access-v2hph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:43:23.970416 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.970388 2548 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-podinfo\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:43:23.970416 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.970413 2548 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-data\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:43:23.970416 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.970422 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2hph\" (UniqueName: \"kubernetes.io/projected/adad1589-7f0b-4401-b2da-398f6b2ce572-kube-api-access-v2hph\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:43:23.970687 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.970430 2548 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-envoy\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:43:23.970687 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:23.970439 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adad1589-7f0b-4401-b2da-398f6b2ce572-istio-token\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:43:24.354933 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:24.354899 2548 generic.go:356] "Generic (PLEG): container finished" podID="adad1589-7f0b-4401-b2da-398f6b2ce572" containerID="762c4fe1c4522693e2cc4c7ec3991dcc17c64ae66a5bdd51d0c2e4d59e4b27e3" exitCode=0 Jul 08 23:43:24.355265 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:24.354963 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" event={"ID":"adad1589-7f0b-4401-b2da-398f6b2ce572","Type":"ContainerDied","Data":"762c4fe1c4522693e2cc4c7ec3991dcc17c64ae66a5bdd51d0c2e4d59e4b27e3"} Jul 08 23:43:24.355265 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:24.354971 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" Jul 08 23:43:24.355265 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:24.355074 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6" event={"ID":"adad1589-7f0b-4401-b2da-398f6b2ce572","Type":"ContainerDied","Data":"44d081e4b20036d48613fde87f75fc786d5359fc045baf6b981d80877615289c"} Jul 08 23:43:24.355265 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:24.355112 2548 scope.go:117] "RemoveContainer" containerID="762c4fe1c4522693e2cc4c7ec3991dcc17c64ae66a5bdd51d0c2e4d59e4b27e3" Jul 08 23:43:24.365294 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:24.365275 2548 scope.go:117] "RemoveContainer" containerID="c97cefcae582ab6b655ab1f9676523a9e7fd228403aec4c849b49dc33de12cca" Jul 08 23:43:24.373050 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:24.373027 2548 scope.go:117] "RemoveContainer" containerID="762c4fe1c4522693e2cc4c7ec3991dcc17c64ae66a5bdd51d0c2e4d59e4b27e3" Jul 08 23:43:24.373294 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:43:24.373275 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"762c4fe1c4522693e2cc4c7ec3991dcc17c64ae66a5bdd51d0c2e4d59e4b27e3\": container with ID starting with 762c4fe1c4522693e2cc4c7ec3991dcc17c64ae66a5bdd51d0c2e4d59e4b27e3 not found: ID does not exist" containerID="762c4fe1c4522693e2cc4c7ec3991dcc17c64ae66a5bdd51d0c2e4d59e4b27e3" Jul 08 23:43:24.373342 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:24.373302 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"762c4fe1c4522693e2cc4c7ec3991dcc17c64ae66a5bdd51d0c2e4d59e4b27e3"} err="failed to get container status \"762c4fe1c4522693e2cc4c7ec3991dcc17c64ae66a5bdd51d0c2e4d59e4b27e3\": rpc error: code = NotFound desc = could not find container \"762c4fe1c4522693e2cc4c7ec3991dcc17c64ae66a5bdd51d0c2e4d59e4b27e3\": container with ID starting with 762c4fe1c4522693e2cc4c7ec3991dcc17c64ae66a5bdd51d0c2e4d59e4b27e3 not found: ID does not exist" Jul 08 23:43:24.373342 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:24.373318 2548 scope.go:117] "RemoveContainer" containerID="c97cefcae582ab6b655ab1f9676523a9e7fd228403aec4c849b49dc33de12cca" Jul 08 23:43:24.373516 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:43:24.373499 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c97cefcae582ab6b655ab1f9676523a9e7fd228403aec4c849b49dc33de12cca\": container with ID starting with c97cefcae582ab6b655ab1f9676523a9e7fd228403aec4c849b49dc33de12cca not found: ID does not exist" containerID="c97cefcae582ab6b655ab1f9676523a9e7fd228403aec4c849b49dc33de12cca" Jul 08 23:43:24.373557 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:24.373524 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c97cefcae582ab6b655ab1f9676523a9e7fd228403aec4c849b49dc33de12cca"} err="failed to get container status \"c97cefcae582ab6b655ab1f9676523a9e7fd228403aec4c849b49dc33de12cca\": rpc error: code = NotFound desc = could not find container \"c97cefcae582ab6b655ab1f9676523a9e7fd228403aec4c849b49dc33de12cca\": container with ID starting with c97cefcae582ab6b655ab1f9676523a9e7fd228403aec4c849b49dc33de12cca not found: ID does not exist" Jul 08 23:43:24.381320 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:24.381302 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6"] Jul 08 23:43:24.385590 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:24.385569 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-45d83d-openshift-default-5764d8d9ff-ckhx6"] Jul 08 23:43:24.828865 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:24.828832 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adad1589-7f0b-4401-b2da-398f6b2ce572" path="/var/lib/kubelet/pods/adad1589-7f0b-4401-b2da-398f6b2ce572/volumes" Jul 08 23:43:25.630388 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:25.630347 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh"] Jul 08 23:43:25.630611 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:25.630557 2548 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" podUID="0969d585-d9fb-495e-9aed-d6de3c07ea11" containerName="istio-proxy" containerID="cri-o://458ff4ce2c9025612b2fce0d2a5168335585d47e69e6679372d2bdc9caf840c9" gracePeriod=30 Jul 08 23:43:26.317435 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:26.317401 2548 patch_prober.go:28] interesting pod/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 23:43:26.317834 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:26.317466 2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" podUID="0969d585-d9fb-495e-9aed-d6de3c07ea11" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 23:43:27.709619 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:27.709556 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nn9bm"] Jul 08 23:43:30.876514 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:30.876486 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:43:31.036706 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.036678 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-envoy\") pod \"0969d585-d9fb-495e-9aed-d6de3c07ea11\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " Jul 08 23:43:31.036857 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.036714 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbp7p\" (UniqueName: \"kubernetes.io/projected/0969d585-d9fb-495e-9aed-d6de3c07ea11-kube-api-access-lbp7p\") pod \"0969d585-d9fb-495e-9aed-d6de3c07ea11\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " Jul 08 23:43:31.036857 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.036748 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-token\") pod \"0969d585-d9fb-495e-9aed-d6de3c07ea11\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " Jul 08 23:43:31.036857 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.036806 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0969d585-d9fb-495e-9aed-d6de3c07ea11-istiod-ca-cert\") pod \"0969d585-d9fb-495e-9aed-d6de3c07ea11\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " Jul 08 23:43:31.036857 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.036835 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-workload-certs\") pod \"0969d585-d9fb-495e-9aed-d6de3c07ea11\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " Jul 08 23:43:31.037097 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.036865 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-data\") pod \"0969d585-d9fb-495e-9aed-d6de3c07ea11\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " Jul 08 23:43:31.037097 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.036892 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-podinfo\") pod \"0969d585-d9fb-495e-9aed-d6de3c07ea11\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " Jul 08 23:43:31.037097 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.036920 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-credential-socket\") pod \"0969d585-d9fb-495e-9aed-d6de3c07ea11\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " Jul 08 23:43:31.037097 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.036954 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-workload-socket\") pod \"0969d585-d9fb-495e-9aed-d6de3c07ea11\" (UID: \"0969d585-d9fb-495e-9aed-d6de3c07ea11\") " Jul 08 23:43:31.037473 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.037306 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0969d585-d9fb-495e-9aed-d6de3c07ea11-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0969d585-d9fb-495e-9aed-d6de3c07ea11" (UID: "0969d585-d9fb-495e-9aed-d6de3c07ea11"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 23:43:31.037473 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.037397 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0969d585-d9fb-495e-9aed-d6de3c07ea11" (UID: "0969d585-d9fb-495e-9aed-d6de3c07ea11"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:43:31.037473 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.037450 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0969d585-d9fb-495e-9aed-d6de3c07ea11" (UID: "0969d585-d9fb-495e-9aed-d6de3c07ea11"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:43:31.037727 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.037623 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0969d585-d9fb-495e-9aed-d6de3c07ea11" (UID: "0969d585-d9fb-495e-9aed-d6de3c07ea11"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:43:31.037788 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.037735 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-data" (OuterVolumeSpecName: "istio-data") pod "0969d585-d9fb-495e-9aed-d6de3c07ea11" (UID: "0969d585-d9fb-495e-9aed-d6de3c07ea11"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:43:31.039048 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.039022 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0969d585-d9fb-495e-9aed-d6de3c07ea11" (UID: "0969d585-d9fb-495e-9aed-d6de3c07ea11"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 23:43:31.039158 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.039027 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-token" (OuterVolumeSpecName: "istio-token") pod "0969d585-d9fb-495e-9aed-d6de3c07ea11" (UID: "0969d585-d9fb-495e-9aed-d6de3c07ea11"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:43:31.039158 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.039079 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0969d585-d9fb-495e-9aed-d6de3c07ea11-kube-api-access-lbp7p" (OuterVolumeSpecName: "kube-api-access-lbp7p") pod "0969d585-d9fb-495e-9aed-d6de3c07ea11" (UID: "0969d585-d9fb-495e-9aed-d6de3c07ea11"). InnerVolumeSpecName "kube-api-access-lbp7p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 23:43:31.039158 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.039152 2548 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0969d585-d9fb-495e-9aed-d6de3c07ea11" (UID: "0969d585-d9fb-495e-9aed-d6de3c07ea11"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 23:43:31.137912 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.137879 2548 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-data\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:43:31.137912 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.137908 2548 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-podinfo\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:43:31.137912 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.137917 2548 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-credential-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:43:31.138140 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.137925 2548 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-workload-socket\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:43:31.138140 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.137934 2548 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-envoy\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:43:31.138140 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.137942 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbp7p\" (UniqueName: \"kubernetes.io/projected/0969d585-d9fb-495e-9aed-d6de3c07ea11-kube-api-access-lbp7p\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:43:31.138140 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.137950 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0969d585-d9fb-495e-9aed-d6de3c07ea11-istio-token\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:43:31.138140 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.137957 2548 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0969d585-d9fb-495e-9aed-d6de3c07ea11-istiod-ca-cert\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:43:31.138140 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.137964 2548 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0969d585-d9fb-495e-9aed-d6de3c07ea11-workload-certs\") on node \"ip-10-0-135-106.ec2.internal\" DevicePath \"\"" Jul 08 23:43:31.393546 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.393452 2548 generic.go:356] "Generic (PLEG): container finished" podID="0969d585-d9fb-495e-9aed-d6de3c07ea11" containerID="458ff4ce2c9025612b2fce0d2a5168335585d47e69e6679372d2bdc9caf840c9" exitCode=0 Jul 08 23:43:31.393546 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.393512 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" Jul 08 23:43:31.393765 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.393540 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" event={"ID":"0969d585-d9fb-495e-9aed-d6de3c07ea11","Type":"ContainerDied","Data":"458ff4ce2c9025612b2fce0d2a5168335585d47e69e6679372d2bdc9caf840c9"} Jul 08 23:43:31.393765 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.393615 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh" event={"ID":"0969d585-d9fb-495e-9aed-d6de3c07ea11","Type":"ContainerDied","Data":"e4864588f18c2a118902a48e6fb2fb95bd284719e5a4dd110da29b57129adf3a"} Jul 08 23:43:31.393765 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.393644 2548 scope.go:117] "RemoveContainer" containerID="458ff4ce2c9025612b2fce0d2a5168335585d47e69e6679372d2bdc9caf840c9" Jul 08 23:43:31.403356 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.403341 2548 scope.go:117] "RemoveContainer" containerID="9132a5cd9ce59e817f6b8e7582c6c8dba26a89f6b13e98834dfaa7550470b60e" Jul 08 23:43:31.411830 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.411812 2548 scope.go:117] "RemoveContainer" containerID="458ff4ce2c9025612b2fce0d2a5168335585d47e69e6679372d2bdc9caf840c9" Jul 08 23:43:31.412079 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:43:31.412063 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"458ff4ce2c9025612b2fce0d2a5168335585d47e69e6679372d2bdc9caf840c9\": container with ID starting with 458ff4ce2c9025612b2fce0d2a5168335585d47e69e6679372d2bdc9caf840c9 not found: ID does not exist" containerID="458ff4ce2c9025612b2fce0d2a5168335585d47e69e6679372d2bdc9caf840c9" Jul 08 23:43:31.412119 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.412087 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"458ff4ce2c9025612b2fce0d2a5168335585d47e69e6679372d2bdc9caf840c9"} err="failed to get container status \"458ff4ce2c9025612b2fce0d2a5168335585d47e69e6679372d2bdc9caf840c9\": rpc error: code = NotFound desc = could not find container \"458ff4ce2c9025612b2fce0d2a5168335585d47e69e6679372d2bdc9caf840c9\": container with ID starting with 458ff4ce2c9025612b2fce0d2a5168335585d47e69e6679372d2bdc9caf840c9 not found: ID does not exist" Jul 08 23:43:31.412119 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.412103 2548 scope.go:117] "RemoveContainer" containerID="9132a5cd9ce59e817f6b8e7582c6c8dba26a89f6b13e98834dfaa7550470b60e" Jul 08 23:43:31.412330 ip-10-0-135-106 kubenswrapper[2548]: E0708 23:43:31.412309 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9132a5cd9ce59e817f6b8e7582c6c8dba26a89f6b13e98834dfaa7550470b60e\": container with ID starting with 9132a5cd9ce59e817f6b8e7582c6c8dba26a89f6b13e98834dfaa7550470b60e not found: ID does not exist" containerID="9132a5cd9ce59e817f6b8e7582c6c8dba26a89f6b13e98834dfaa7550470b60e" Jul 08 23:43:31.412365 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.412340 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9132a5cd9ce59e817f6b8e7582c6c8dba26a89f6b13e98834dfaa7550470b60e"} err="failed to get container status \"9132a5cd9ce59e817f6b8e7582c6c8dba26a89f6b13e98834dfaa7550470b60e\": rpc error: code = NotFound desc = could not find container \"9132a5cd9ce59e817f6b8e7582c6c8dba26a89f6b13e98834dfaa7550470b60e\": container with ID starting with 9132a5cd9ce59e817f6b8e7582c6c8dba26a89f6b13e98834dfaa7550470b60e not found: ID does not exist" Jul 08 23:43:31.418046 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.418027 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh"] Jul 08 23:43:31.421623 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.421589 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-59ce67-openshift-default-d8d4d4cbc-b9fkh"] Jul 08 23:43:31.836316 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.836288 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ccc7894f9-x75gt_64fb8ced-2781-4225-ac84-f16f956e4061/copy-manifests/0.log" Jul 08 23:43:31.849300 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:31.849275 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ccc7894f9-x75gt_64fb8ced-2781-4225-ac84-f16f956e4061/manager/0.log" Jul 08 23:43:32.833442 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:32.829885 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0969d585-d9fb-495e-9aed-d6de3c07ea11" path="/var/lib/kubelet/pods/0969d585-d9fb-495e-9aed-d6de3c07ea11/volumes" Jul 08 23:43:33.056893 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:33.056861 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d_df566e6b-9d68-4398-a353-2c510cf87f6d/extract/0.log" Jul 08 23:43:33.062813 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:33.062788 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d_df566e6b-9d68-4398-a353-2c510cf87f6d/util/0.log" Jul 08 23:43:33.068852 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:33.068833 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d_df566e6b-9d68-4398-a353-2c510cf87f6d/pull/0.log" Jul 08 23:43:33.178728 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:33.178665 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7_85157b14-97af-430b-9a83-295bfd02218e/extract/0.log" Jul 08 23:43:33.186147 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:33.186127 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7_85157b14-97af-430b-9a83-295bfd02218e/util/0.log" Jul 08 23:43:33.192100 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:33.192082 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7_85157b14-97af-430b-9a83-295bfd02218e/pull/0.log" Jul 08 23:43:33.303916 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:33.303889 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv_32db00ac-9c9c-40a5-98bd-d995725ab58b/util/0.log" Jul 08 23:43:33.310294 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:33.310272 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv_32db00ac-9c9c-40a5-98bd-d995725ab58b/pull/0.log" Jul 08 23:43:33.316486 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:33.316466 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv_32db00ac-9c9c-40a5-98bd-d995725ab58b/extract/0.log" Jul 08 23:43:33.423803 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:33.423778 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5_2d797d5d-08e0-4ffd-9853-941bb75524cb/util/0.log" Jul 08 23:43:33.430004 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:33.429942 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5_2d797d5d-08e0-4ffd-9853-941bb75524cb/pull/0.log" Jul 08 23:43:33.440529 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:33.440500 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5_2d797d5d-08e0-4ffd-9853-941bb75524cb/extract/0.log" Jul 08 23:43:33.679487 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:33.679459 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-tchdl_ad3c68fb-f770-4f70-b7b4-8016f5262f74/manager/0.log" Jul 08 23:43:34.006445 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:34.006418 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-r4xs4_315f4cc0-5159-4ac6-9a39-9d4175af042a/registry-server/0.log" Jul 08 23:43:34.125726 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:34.125697 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-nvdb9_74513fe2-178e-4104-a6d7-441a1ceaf02d/manager/0.log" Jul 08 23:43:34.235456 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:34.235424 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-nn9bm_8efd0329-b2bf-4129-ac63-c4c87a831472/limitador/0.log" Jul 08 23:43:34.693264 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:34.693234 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4br7p9_104e3c4c-f2de-4b0e-8b82-c69872e92066/istio-proxy/0.log" Jul 08 23:43:35.147447 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:35.147419 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qlwdl_a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c/istio-proxy/1.log" Jul 08 23:43:40.289399 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.289364 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rlvz5/must-gather-kbkmq"] Jul 08 23:43:40.289792 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.289739 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0969d585-d9fb-495e-9aed-d6de3c07ea11" containerName="istio-proxy" Jul 08 23:43:40.289792 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.289751 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="0969d585-d9fb-495e-9aed-d6de3c07ea11" containerName="istio-proxy" Jul 08 23:43:40.289792 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.289768 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0969d585-d9fb-495e-9aed-d6de3c07ea11" containerName="istio-proxy" Jul 08 23:43:40.289792 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.289773 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="0969d585-d9fb-495e-9aed-d6de3c07ea11" containerName="istio-proxy" Jul 08 23:43:40.289792 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.289787 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab66be46-5c0d-4475-a909-60af39531ce8" containerName="test-curl-7671-140539381375456" Jul 08 23:43:40.289792 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.289793 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab66be46-5c0d-4475-a909-60af39531ce8" containerName="test-curl-7671-140539381375456" Jul 08 23:43:40.289974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.289802 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adad1589-7f0b-4401-b2da-398f6b2ce572" containerName="istio-proxy" Jul 08 23:43:40.289974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.289807 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="adad1589-7f0b-4401-b2da-398f6b2ce572" containerName="istio-proxy" Jul 08 23:43:40.289974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.289815 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adad1589-7f0b-4401-b2da-398f6b2ce572" containerName="istio-proxy" Jul 08 23:43:40.289974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.289820 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="adad1589-7f0b-4401-b2da-398f6b2ce572" containerName="istio-proxy" Jul 08 23:43:40.289974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.289870 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="adad1589-7f0b-4401-b2da-398f6b2ce572" containerName="istio-proxy" Jul 08 23:43:40.289974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.289877 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="adad1589-7f0b-4401-b2da-398f6b2ce572" containerName="istio-proxy" Jul 08 23:43:40.289974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.289883 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab66be46-5c0d-4475-a909-60af39531ce8" containerName="test-curl-7671-140539381375456" Jul 08 23:43:40.289974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.289889 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="0969d585-d9fb-495e-9aed-d6de3c07ea11" containerName="istio-proxy" Jul 08 23:43:40.289974 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.289895 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="0969d585-d9fb-495e-9aed-d6de3c07ea11" containerName="istio-proxy" Jul 08 23:43:40.293024 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.293001 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rlvz5/must-gather-kbkmq" Jul 08 23:43:40.295574 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.295553 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rlvz5\"/\"openshift-service-ca.crt\"" Jul 08 23:43:40.295862 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.295848 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rlvz5\"/\"kube-root-ca.crt\"" Jul 08 23:43:40.296808 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.296792 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rlvz5\"/\"default-dockercfg-hrrrl\"" Jul 08 23:43:40.304187 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.304036 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rlvz5/must-gather-kbkmq"] Jul 08 23:43:40.315062 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.315040 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4f55cc8c-e2be-4cd9-a66d-b8e26f260ba3-must-gather-output\") pod \"must-gather-kbkmq\" (UID: \"4f55cc8c-e2be-4cd9-a66d-b8e26f260ba3\") " pod="openshift-must-gather-rlvz5/must-gather-kbkmq" Jul 08 23:43:40.315161 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.315098 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shnkq\" (UniqueName: \"kubernetes.io/projected/4f55cc8c-e2be-4cd9-a66d-b8e26f260ba3-kube-api-access-shnkq\") pod \"must-gather-kbkmq\" (UID: \"4f55cc8c-e2be-4cd9-a66d-b8e26f260ba3\") " pod="openshift-must-gather-rlvz5/must-gather-kbkmq" Jul 08 23:43:40.415531 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.415498 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shnkq\" (UniqueName: \"kubernetes.io/projected/4f55cc8c-e2be-4cd9-a66d-b8e26f260ba3-kube-api-access-shnkq\") pod \"must-gather-kbkmq\" (UID: \"4f55cc8c-e2be-4cd9-a66d-b8e26f260ba3\") " pod="openshift-must-gather-rlvz5/must-gather-kbkmq" Jul 08 23:43:40.415715 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.415557 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4f55cc8c-e2be-4cd9-a66d-b8e26f260ba3-must-gather-output\") pod \"must-gather-kbkmq\" (UID: \"4f55cc8c-e2be-4cd9-a66d-b8e26f260ba3\") " pod="openshift-must-gather-rlvz5/must-gather-kbkmq" Jul 08 23:43:40.415912 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.415892 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4f55cc8c-e2be-4cd9-a66d-b8e26f260ba3-must-gather-output\") pod \"must-gather-kbkmq\" (UID: \"4f55cc8c-e2be-4cd9-a66d-b8e26f260ba3\") " pod="openshift-must-gather-rlvz5/must-gather-kbkmq" Jul 08 23:43:40.429911 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.429885 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shnkq\" (UniqueName: \"kubernetes.io/projected/4f55cc8c-e2be-4cd9-a66d-b8e26f260ba3-kube-api-access-shnkq\") pod \"must-gather-kbkmq\" (UID: \"4f55cc8c-e2be-4cd9-a66d-b8e26f260ba3\") " pod="openshift-must-gather-rlvz5/must-gather-kbkmq" Jul 08 23:43:40.609829 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.609738 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rlvz5/must-gather-kbkmq" Jul 08 23:43:40.745615 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:40.745562 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rlvz5/must-gather-kbkmq"] Jul 08 23:43:40.746927 ip-10-0-135-106 kubenswrapper[2548]: W0708 23:43:40.746895 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f55cc8c_e2be_4cd9_a66d_b8e26f260ba3.slice/crio-16c06a5a4b4dc52159f9622216bf509477318f94cadebaf970eb7f8b2e121ae3 WatchSource:0}: Error finding container 16c06a5a4b4dc52159f9622216bf509477318f94cadebaf970eb7f8b2e121ae3: Status 404 returned error can't find the container with id 16c06a5a4b4dc52159f9622216bf509477318f94cadebaf970eb7f8b2e121ae3 Jul 08 23:43:41.447898 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:41.447851 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rlvz5/must-gather-kbkmq" event={"ID":"4f55cc8c-e2be-4cd9-a66d-b8e26f260ba3","Type":"ContainerStarted","Data":"16c06a5a4b4dc52159f9622216bf509477318f94cadebaf970eb7f8b2e121ae3"} Jul 08 23:43:42.456540 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:42.456492 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rlvz5/must-gather-kbkmq" event={"ID":"4f55cc8c-e2be-4cd9-a66d-b8e26f260ba3","Type":"ContainerStarted","Data":"b0940b15eb17099d3cbe1ff234a02abe946d9145cb873d35cc73c121ec559d04"} Jul 08 23:43:42.457027 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:42.456556 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rlvz5/must-gather-kbkmq" event={"ID":"4f55cc8c-e2be-4cd9-a66d-b8e26f260ba3","Type":"ContainerStarted","Data":"da12a9d88d0ade2b04f2d9fbbdfd99a8998c44a79a8d0713cfeb9e0dc16c6840"} Jul 08 23:43:42.474078 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:42.474020 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rlvz5/must-gather-kbkmq" podStartSLOduration=1.665417158 podStartE2EDuration="2.473997992s" totalImagesPullingTime="808.580834ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:43:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:43:42.473921297 +0000 UTC m=+2444.234416365" watchObservedRunningTime="2026-07-08 23:43:42.473997992 +0000 UTC m=+2444.234493043" Jul 08 23:43:43.084566 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:43.084532 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dwldq_213525b4-d967-437a-9c90-d40f2190a5a5/global-pull-secret-syncer/0.log" Jul 08 23:43:43.290201 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:43.290175 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-x6gwk_aa3acccd-8292-46b0-b34b-d6ecba8fc595/konnectivity-agent/0.log" Jul 08 23:43:43.338939 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:43.338859 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-106.ec2.internal_c95e074b8f38074e93586ba30700e2a9/haproxy/0.log" Jul 08 23:43:47.591398 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:47.591354 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d_df566e6b-9d68-4398-a353-2c510cf87f6d/extract/0.log" Jul 08 23:43:47.618824 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:47.618747 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d_df566e6b-9d68-4398-a353-2c510cf87f6d/util/0.log" Jul 08 23:43:47.647865 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:47.647832 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cr24d_df566e6b-9d68-4398-a353-2c510cf87f6d/pull/0.log" Jul 08 23:43:47.677407 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:47.677381 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7_85157b14-97af-430b-9a83-295bfd02218e/extract/0.log" Jul 08 23:43:47.714852 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:47.714819 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7_85157b14-97af-430b-9a83-295bfd02218e/util/0.log" Jul 08 23:43:47.739457 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:47.739431 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fbrg7_85157b14-97af-430b-9a83-295bfd02218e/pull/0.log" Jul 08 23:43:47.772203 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:47.772176 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv_32db00ac-9c9c-40a5-98bd-d995725ab58b/extract/0.log" Jul 08 23:43:47.812188 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:47.812156 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv_32db00ac-9c9c-40a5-98bd-d995725ab58b/util/0.log" Jul 08 23:43:47.838104 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:47.838059 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p8gfv_32db00ac-9c9c-40a5-98bd-d995725ab58b/pull/0.log" Jul 08 23:43:47.873720 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:47.873590 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5_2d797d5d-08e0-4ffd-9853-941bb75524cb/extract/0.log" Jul 08 23:43:47.898530 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:47.898506 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5_2d797d5d-08e0-4ffd-9853-941bb75524cb/util/0.log" Jul 08 23:43:47.922113 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:47.922061 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef168jv5_2d797d5d-08e0-4ffd-9853-941bb75524cb/pull/0.log" Jul 08 23:43:48.473074 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:48.473048 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-tchdl_ad3c68fb-f770-4f70-b7b4-8016f5262f74/manager/0.log" Jul 08 23:43:48.600362 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:48.600337 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-r4xs4_315f4cc0-5159-4ac6-9a39-9d4175af042a/registry-server/0.log" Jul 08 23:43:48.757353 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:48.756631 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-nvdb9_74513fe2-178e-4104-a6d7-441a1ceaf02d/manager/0.log" Jul 08 23:43:48.782006 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:48.781976 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-nn9bm_8efd0329-b2bf-4129-ac63-c4c87a831472/limitador/0.log" Jul 08 23:43:50.850966 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:50.850934 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-ncgc8_eccab368-9199-4510-a77e-6909f8116c1e/kube-state-metrics/0.log" Jul 08 23:43:50.871806 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:50.871779 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-ncgc8_eccab368-9199-4510-a77e-6909f8116c1e/kube-rbac-proxy-main/0.log" Jul 08 23:43:50.897509 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:50.897484 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-ncgc8_eccab368-9199-4510-a77e-6909f8116c1e/kube-rbac-proxy-self/0.log" Jul 08 23:43:50.962832 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:50.962804 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-bkr4d_1cf33f7c-f467-48d7-99d7-2a93d7b55583/monitoring-plugin/0.log" Jul 08 23:43:50.999264 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:50.999235 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6dl92_13324038-f77f-43d4-9b13-7c8935e64c5b/node-exporter/0.log" Jul 08 23:43:51.025957 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:51.025931 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6dl92_13324038-f77f-43d4-9b13-7c8935e64c5b/kube-rbac-proxy/0.log" Jul 08 23:43:51.049018 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:51.048991 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6dl92_13324038-f77f-43d4-9b13-7c8935e64c5b/init-textfile/0.log" Jul 08 23:43:51.242702 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:51.242667 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fp2bp_88e9bf6f-ea85-45ee-970a-a7e4632e477d/kube-rbac-proxy-main/0.log" Jul 08 23:43:51.267390 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:51.267363 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fp2bp_88e9bf6f-ea85-45ee-970a-a7e4632e477d/kube-rbac-proxy-self/0.log" Jul 08 23:43:51.290187 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:51.290145 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fp2bp_88e9bf6f-ea85-45ee-970a-a7e4632e477d/openshift-state-metrics/0.log" Jul 08 23:43:51.579676 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:51.579575 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5fd4fbfbc7-7vvn7_0d9d05b2-43f2-4dc1-8464-1de07362c7e4/telemeter-client/0.log" Jul 08 23:43:51.603971 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:51.603945 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5fd4fbfbc7-7vvn7_0d9d05b2-43f2-4dc1-8464-1de07362c7e4/reload/0.log" Jul 08 23:43:51.626748 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:51.626713 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5fd4fbfbc7-7vvn7_0d9d05b2-43f2-4dc1-8464-1de07362c7e4/kube-rbac-proxy/0.log" Jul 08 23:43:53.129400 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.129356 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg"] Jul 08 23:43:53.137120 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.137081 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" Jul 08 23:43:53.140577 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.140549 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg"] Jul 08 23:43:53.262259 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.262213 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/886d0f8c-5856-4ecc-90a7-3f6473a2f2f2-podres\") pod \"perf-node-gather-daemonset-t74qg\" (UID: \"886d0f8c-5856-4ecc-90a7-3f6473a2f2f2\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" Jul 08 23:43:53.262454 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.262355 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/886d0f8c-5856-4ecc-90a7-3f6473a2f2f2-proc\") pod \"perf-node-gather-daemonset-t74qg\" (UID: \"886d0f8c-5856-4ecc-90a7-3f6473a2f2f2\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" Jul 08 23:43:53.262454 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.262397 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7gjm\" (UniqueName: \"kubernetes.io/projected/886d0f8c-5856-4ecc-90a7-3f6473a2f2f2-kube-api-access-j7gjm\") pod \"perf-node-gather-daemonset-t74qg\" (UID: \"886d0f8c-5856-4ecc-90a7-3f6473a2f2f2\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" Jul 08 23:43:53.262454 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.262441 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/886d0f8c-5856-4ecc-90a7-3f6473a2f2f2-sys\") pod \"perf-node-gather-daemonset-t74qg\" (UID: \"886d0f8c-5856-4ecc-90a7-3f6473a2f2f2\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" Jul 08 23:43:53.262668 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.262468 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/886d0f8c-5856-4ecc-90a7-3f6473a2f2f2-lib-modules\") pod \"perf-node-gather-daemonset-t74qg\" (UID: \"886d0f8c-5856-4ecc-90a7-3f6473a2f2f2\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" Jul 08 23:43:53.363915 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.363863 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/886d0f8c-5856-4ecc-90a7-3f6473a2f2f2-proc\") pod \"perf-node-gather-daemonset-t74qg\" (UID: \"886d0f8c-5856-4ecc-90a7-3f6473a2f2f2\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" Jul 08 23:43:53.364119 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.363937 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7gjm\" (UniqueName: \"kubernetes.io/projected/886d0f8c-5856-4ecc-90a7-3f6473a2f2f2-kube-api-access-j7gjm\") pod \"perf-node-gather-daemonset-t74qg\" (UID: \"886d0f8c-5856-4ecc-90a7-3f6473a2f2f2\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" Jul 08 23:43:53.364119 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.363984 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/886d0f8c-5856-4ecc-90a7-3f6473a2f2f2-sys\") pod \"perf-node-gather-daemonset-t74qg\" (UID: \"886d0f8c-5856-4ecc-90a7-3f6473a2f2f2\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" Jul 08 23:43:53.364119 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.364012 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/886d0f8c-5856-4ecc-90a7-3f6473a2f2f2-lib-modules\") pod \"perf-node-gather-daemonset-t74qg\" (UID: \"886d0f8c-5856-4ecc-90a7-3f6473a2f2f2\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" Jul 08 23:43:53.364119 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.364048 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/886d0f8c-5856-4ecc-90a7-3f6473a2f2f2-podres\") pod \"perf-node-gather-daemonset-t74qg\" (UID: \"886d0f8c-5856-4ecc-90a7-3f6473a2f2f2\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" Jul 08 23:43:53.364341 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.364282 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/886d0f8c-5856-4ecc-90a7-3f6473a2f2f2-podres\") pod \"perf-node-gather-daemonset-t74qg\" (UID: \"886d0f8c-5856-4ecc-90a7-3f6473a2f2f2\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" Jul 08 23:43:53.364397 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.364346 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/886d0f8c-5856-4ecc-90a7-3f6473a2f2f2-proc\") pod \"perf-node-gather-daemonset-t74qg\" (UID: \"886d0f8c-5856-4ecc-90a7-3f6473a2f2f2\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" Jul 08 23:43:53.364741 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.364715 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/886d0f8c-5856-4ecc-90a7-3f6473a2f2f2-sys\") pod \"perf-node-gather-daemonset-t74qg\" (UID: \"886d0f8c-5856-4ecc-90a7-3f6473a2f2f2\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" Jul 08 23:43:53.364858 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.364842 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/886d0f8c-5856-4ecc-90a7-3f6473a2f2f2-lib-modules\") pod \"perf-node-gather-daemonset-t74qg\" (UID: \"886d0f8c-5856-4ecc-90a7-3f6473a2f2f2\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" Jul 08 23:43:53.373354 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.373308 2548 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7gjm\" (UniqueName: \"kubernetes.io/projected/886d0f8c-5856-4ecc-90a7-3f6473a2f2f2-kube-api-access-j7gjm\") pod \"perf-node-gather-daemonset-t74qg\" (UID: \"886d0f8c-5856-4ecc-90a7-3f6473a2f2f2\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" Jul 08 23:43:53.463386 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.463297 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" Jul 08 23:43:53.638702 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:53.638529 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg"] Jul 08 23:43:54.065544 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:54.065459 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-b4b74587c-ccgb8_052f84cf-54ae-4565-a35a-3d7ed3a53f63/console/0.log" Jul 08 23:43:54.102138 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:54.102106 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-4686n_091053ae-91a5-41d2-93a5-ecedb8b381f8/download-server/0.log" Jul 08 23:43:54.550993 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:54.550956 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" event={"ID":"886d0f8c-5856-4ecc-90a7-3f6473a2f2f2","Type":"ContainerStarted","Data":"36ec3e95730e628e6f67d1ce06a052371eff2f408c5d66f789f1a739a644f29f"} Jul 08 23:43:54.551398 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:54.551009 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" event={"ID":"886d0f8c-5856-4ecc-90a7-3f6473a2f2f2","Type":"ContainerStarted","Data":"83e387c40a3be0dad593377373489562ffac96c110194063677990ee1e49b86d"} Jul 08 23:43:54.551398 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:54.551065 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" Jul 08 23:43:54.568112 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:54.568063 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" podStartSLOduration=1.5680465749999999 podStartE2EDuration="1.568046575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:43:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:43:54.567215008 +0000 UTC m=+2456.327710073" watchObservedRunningTime="2026-07-08 23:43:54.568046575 +0000 UTC m=+2456.328541606" Jul 08 23:43:55.734397 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:55.734369 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jb42p_f9a58df3-ab7d-4b82-9e17-b7b140727a4b/dns/0.log" Jul 08 23:43:55.757527 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:55.757505 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jb42p_f9a58df3-ab7d-4b82-9e17-b7b140727a4b/kube-rbac-proxy/0.log" Jul 08 23:43:55.940689 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:55.940652 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wnnrr_c8cf1ec4-b117-47ba-ad3e-0c0e147a2a28/dns-node-resolver/0.log" Jul 08 23:43:56.465305 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:56.465274 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7cc7b69d54-6vkwp_1fc2cc5c-7632-4171-9813-d13c9b038424/registry/0.log" Jul 08 23:43:56.523052 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:56.523000 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mt56m_44817e25-f0f3-44fc-ab14-f1cc0cddd89e/node-ca/0.log" Jul 08 23:43:57.494937 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:57.494905 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4br7p9_104e3c4c-f2de-4b0e-8b82-c69872e92066/istio-proxy/0.log" Jul 08 23:43:57.754969 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:57.754884 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qlwdl_a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c/istio-proxy/0.log" Jul 08 23:43:57.784628 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:57.784582 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qlwdl_a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c/istio-proxy/1.log" Jul 08 23:43:58.719472 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:58.719442 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hsjss_b8fae214-eb17-4ee8-b4d2-de209a11ad64/serve-healthcheck-canary/0.log" Jul 08 23:43:59.416509 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:59.416444 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x4nvj_577bedd6-568d-4e24-ab8e-0ed9ae106385/kube-rbac-proxy/0.log" Jul 08 23:43:59.439335 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:59.439308 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x4nvj_577bedd6-568d-4e24-ab8e-0ed9ae106385/exporter/0.log" Jul 08 23:43:59.464865 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:43:59.464840 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x4nvj_577bedd6-568d-4e24-ab8e-0ed9ae106385/extractor/0.log" Jul 08 23:44:00.568065 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:44:00.568034 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-t74qg" Jul 08 23:44:02.332568 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:44:02.326314 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ccc7894f9-x75gt_64fb8ced-2781-4225-ac84-f16f956e4061/manager/0.log" Jul 08 23:44:02.349926 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:44:02.349900 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ccc7894f9-x75gt_64fb8ced-2781-4225-ac84-f16f956e4061/copy-manifests/0.log" Jul 08 23:44:03.294676 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:44:03.294647 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4br7p9_104e3c4c-f2de-4b0e-8b82-c69872e92066/istio-proxy/0.log" Jul 08 23:44:03.437923 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:44:03.437824 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qlwdl_a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c/istio-proxy/0.log" Jul 08 23:44:03.474677 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:44:03.474642 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qlwdl_a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c/istio-proxy/1.log" Jul 08 23:44:03.701451 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:44:03.701381 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-65f9849997-sg85f_3096021c-13fa-4c6d-90f4-d248ddc3403f/manager/0.log" Jul 08 23:44:03.728711 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:44:03.728679 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-9wv9s_792f3943-1813-4ede-9320-6555ae2de4dc/openshift-lws-operator/0.log" Jul 08 23:44:04.745809 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:44:04.745777 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4br7p9_104e3c4c-f2de-4b0e-8b82-c69872e92066/istio-proxy/0.log" Jul 08 23:44:04.905544 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:44:04.905504 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qlwdl_a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c/istio-proxy/0.log" Jul 08 23:44:04.941702 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:44:04.941671 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qlwdl_a6a80db1-8a77-4f08-ae4c-7ef02f7cb01c/istio-proxy/1.log" Jul 08 23:44:10.065912 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:44:10.065872 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nwsqm_58690598-bf03-4609-bc53-3d4b5edca7fc/kube-multus-additional-cni-plugins/0.log" Jul 08 23:44:10.088428 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:44:10.088405 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nwsqm_58690598-bf03-4609-bc53-3d4b5edca7fc/egress-router-binary-copy/0.log" Jul 08 23:44:10.114630 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:44:10.114591 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nwsqm_58690598-bf03-4609-bc53-3d4b5edca7fc/cni-plugins/0.log" Jul 08 23:44:10.135900 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:44:10.135877 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nwsqm_58690598-bf03-4609-bc53-3d4b5edca7fc/bond-cni-plugin/0.log" Jul 08 23:44:10.157660 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:44:10.157635 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nwsqm_58690598-bf03-4609-bc53-3d4b5edca7fc/routeoverride-cni/0.log" Jul 08 23:44:10.181728 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:44:10.181708 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nwsqm_58690598-bf03-4609-bc53-3d4b5edca7fc/whereabouts-cni-bincopy/0.log" Jul 08 23:44:10.205676 ip-10-0-135-106 kubenswrapper[2548]: I0708 23:44:10.205654 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nwsqm_58690598-bf03-4609-bc53-3d4b5edca7fc/whereabouts-cni/0.log"