Sep 06 10:28:09.430850 ip-10-0-133-153 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 06 10:28:09.430861 ip-10-0-133-153 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 06 10:28:09.430869 ip-10-0-133-153 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 06 10:28:09.431102 ip-10-0-133-153 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 06 10:28:19.203067 ip-10-0-133-153 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Sep 06 10:28:19.203083 ip-10-0-133-153 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot d906151c0f5e45999ff6052500090b9b --
Sep 06 10:29:46.102794 ip-10-0-133-153 systemd[1]: Starting Kubernetes Kubelet...
Sep 06 10:29:46.547903 ip-10-0-133-153 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 06 10:29:46.547903 ip-10-0-133-153 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 06 10:29:46.547903 ip-10-0-133-153 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 06 10:29:46.547903 ip-10-0-133-153 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 06 10:29:46.547903 ip-10-0-133-153 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 06 10:29:46.550889 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.550800    2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 06 10:29:46.556733 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556705    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 10:29:46.556733 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556726    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 10:29:46.556733 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556729    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 10:29:46.556733 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556733    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 10:29:46.556733 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556735    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 10:29:46.556733 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556738    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 10:29:46.556733 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556741    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556744    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556747    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556749    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556752    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556754    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556757    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556760    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556763    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556765    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556769    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556772    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556774    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556777    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556780    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556782    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556785    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556788    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556791    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556793    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 10:29:46.556996 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556796    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556799    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556801    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556804    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556809    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556812    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556815    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556818    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556820    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556823    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556825    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556828    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556831    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556834    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556837    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556840    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556843    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556846    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556848    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556851    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 10:29:46.557497 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556853    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 10:29:46.558025 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556856    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 10:29:46.558025 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556859    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 10:29:46.558025 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556861    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 10:29:46.558025 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556864    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 10:29:46.558025 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556867    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 10:29:46.558025 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556869    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 10:29:46.558025 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556872    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 10:29:46.558025 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556874    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 10:29:46.558025 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556879    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 10:29:46.558025 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556884    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 10:29:46.558025 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556887    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 10:29:46.558025 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556889    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 10:29:46.558025 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556892    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 10:29:46.558025 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556896    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 10:29:46.558025 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556898    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 10:29:46.558025 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556901    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 10:29:46.558025 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556903    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 10:29:46.558025 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556906    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 10:29:46.558025 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556909    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 10:29:46.558511 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556912    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 10:29:46.558511 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556916    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 10:29:46.558511 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556920    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 10:29:46.558511 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556924    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 10:29:46.558511 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556927    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 10:29:46.558511 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556930    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 10:29:46.558511 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556934    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 10:29:46.558511 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556939    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 10:29:46.558511 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556942    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 10:29:46.558511 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556945    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 10:29:46.558511 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556947    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 10:29:46.558511 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556950    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 10:29:46.558511 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556953    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 10:29:46.558511 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556955    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 10:29:46.558511 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556958    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 10:29:46.558511 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556961    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 10:29:46.558511 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556964    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 10:29:46.558511 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556968    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 10:29:46.558511 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556971    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556974    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556977    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556980    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556983    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556986    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556989    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556991    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556994    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556996    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.556999    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557002    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557004    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557007    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557009    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557763    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557771    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557775    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557779    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557782    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 10:29:46.558972 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557786    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 10:29:46.559441 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557789    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 10:29:46.559441 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557792    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 10:29:46.559441 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557794    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 10:29:46.559441 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557797    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 10:29:46.559441 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557800    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 10:29:46.559441 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557802    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 10:29:46.559441 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557805    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 10:29:46.559441 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557807    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 10:29:46.559441 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557810    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 10:29:46.559441 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557814    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 10:29:46.559441 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557818    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 10:29:46.559441 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557822    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 10:29:46.559441 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557825    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 10:29:46.559441 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557828    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 10:29:46.559441 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557830    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 10:29:46.559441 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557833    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 10:29:46.559441 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557836    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 10:29:46.559441 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557838    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 10:29:46.559441 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557841    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 10:29:46.559924 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557843    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 10:29:46.559924 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557846    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 10:29:46.559924 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557849    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 10:29:46.559924 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557851    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 10:29:46.559924 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557854    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 10:29:46.559924 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557857    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 10:29:46.559924 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557859    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 10:29:46.559924 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557862    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 10:29:46.559924 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557867    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 10:29:46.559924 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557870    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 10:29:46.559924 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557874    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 10:29:46.559924 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557877    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 10:29:46.559924 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557880    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 10:29:46.559924 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557882    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 10:29:46.559924 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557885    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 10:29:46.559924 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557887    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 10:29:46.559924 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557890    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 10:29:46.559924 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557893    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 10:29:46.559924 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557895    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557898    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557901    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557903    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557906    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557909    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557911    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557915    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557917    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557920    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557922    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557925    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557927    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557930    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557932    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557935    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557938    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557940    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557943    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557945    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 10:29:46.560424 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557948    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557951    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557953    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557956    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557960    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557963    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557965    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557968    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557970    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557973    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557976    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557981    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557983    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557986    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557988    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557991    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557993    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557995    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.557998    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558000    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 10:29:46.560910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558004    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558006    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558009    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558012    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558015    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558017    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558020    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558022    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558025    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558027    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558032    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558035    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558038    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558040    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558042    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558045    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558416    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558420    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558423    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558426    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 10:29:46.561412 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558429    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 10:29:46.561910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558431    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 10:29:46.561910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558434    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 10:29:46.561910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558437    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 10:29:46.561910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558440    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 10:29:46.561910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558442    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 10:29:46.561910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558445    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 10:29:46.561910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558447    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 10:29:46.561910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558450    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 10:29:46.561910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558454    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 10:29:46.561910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558478    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 10:29:46.561910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558482    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 10:29:46.561910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558486    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 10:29:46.561910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558489    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 10:29:46.561910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558492    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 10:29:46.561910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558494    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 10:29:46.561910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558497    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 10:29:46.561910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558501    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 10:29:46.561910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558505    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 10:29:46.561910 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558508    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 10:29:46.562385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558510    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 10:29:46.562385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558513    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 10:29:46.562385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558518    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 10:29:46.562385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558523    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 10:29:46.562385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558526    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 10:29:46.562385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558529    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 10:29:46.562385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558533    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 10:29:46.562385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558536    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 10:29:46.562385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558539    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 10:29:46.562385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558542    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 10:29:46.562385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558544    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 10:29:46.562385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558553    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 10:29:46.562385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558556    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 10:29:46.562385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558559    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 10:29:46.562385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558561    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 10:29:46.562385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558564    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 10:29:46.562385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558567    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 10:29:46.562385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558569    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 10:29:46.562385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558572    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558574    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558577    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558579    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558582    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558585    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558587    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558590    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558593    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558596    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558598    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558601    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558604    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558606    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558609    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558611    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558614    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558617    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558621    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558624    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 10:29:46.562877 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558626    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558629    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558632    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558634    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558637    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558639    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558642    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558645    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558647    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558650    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558652    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558656    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558659    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558661    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558664    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558666    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558669    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558671    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558674    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558676    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 10:29:46.563373 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558680    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558682    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558684    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558687    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558689    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558692    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558694    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558697    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558699    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558702    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558706    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558709    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558711    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558714    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558716    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558719    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.558721    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558799    2557 flags.go:64] FLAG: --address="0.0.0.0"
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558843    2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558851    2557 flags.go:64] FLAG: --anonymous-auth="true"
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558855    2557 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 06 10:29:46.563896 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558859    2557 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558862    2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558867    2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558871    2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558874    2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558877    2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558881    2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558885    2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558888    2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558890    2557 flags.go:64] FLAG: --cgroup-root=""
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558893    2557 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558897    2557 flags.go:64] FLAG: --client-ca-file=""
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558900    2557 flags.go:64] FLAG: --cloud-provider="external"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558903    2557 flags.go:64] FLAG: --cluster-dns="[]"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558907    2557 flags.go:64] FLAG: --cluster-domain=""
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558910    2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558913    2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558916    2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558919    2557 flags.go:64] FLAG: --container-log-max-files="5"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558923    2557 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558927    2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558931    2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558935    2557 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558938    2557 flags.go:64] FLAG: --contention-profiling="false"
Sep 06 10:29:46.564400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558941    2557 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558944    2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558947    2557 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558950    2557 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558954    2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558957    2557 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558960    2557 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558963    2557 flags.go:64] FLAG: --enable-load-reader="false"
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558966    2557 flags.go:64] FLAG: --enable-server="true"
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558969    2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558973    2557 flags.go:64] FLAG: --event-burst="100"
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558977    2557 flags.go:64] FLAG: --event-qps="50"
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558980    2557 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558983    2557 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558986    2557 flags.go:64] FLAG: --eviction-hard=""
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558990    2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558993    2557 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558996    2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.558999    2557 flags.go:64] FLAG: --eviction-soft=""
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559002    2557 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559005    2557 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559008    2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559011    2557 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559014    2557 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559017    2557 flags.go:64] FLAG: --fail-swap-on="true"
Sep 06 10:29:46.564986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559020    2557 flags.go:64] FLAG: --feature-gates=""
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559023    2557 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559026    2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559029    2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559035    2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559038    2557 flags.go:64] FLAG: --healthz-port="10248"
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559040    2557 flags.go:64] FLAG: --help="false"
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559043    2557 flags.go:64] FLAG: --hostname-override="ip-10-0-133-153.ec2.internal"
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559046    2557 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559049    2557 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559052    2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559055    2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559058    2557 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559061    2557 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559064    2557 flags.go:64] FLAG: --image-service-endpoint=""
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559067    2557 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559069    2557 flags.go:64] FLAG: --kube-api-burst="100"
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559072    2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559075    2557 flags.go:64] FLAG: --kube-api-qps="50"
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559078    2557 flags.go:64] FLAG: --kube-reserved=""
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559081    2557 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559083    2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559086    2557 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559089    2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 06 10:29:46.565754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559092    2557 flags.go:64] FLAG: --lock-file=""
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559094    2557 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559097    2557 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559100    2557 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559105    2557 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559109    2557 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559111    2557 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559114    2557 flags.go:64] FLAG: --logging-format="text"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559117    2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559120    2557 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559124    2557 flags.go:64] FLAG: --manifest-url=""
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559127    2557 flags.go:64] FLAG: --manifest-url-header=""
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559132    2557 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559135    2557 flags.go:64] FLAG: --max-open-files="1000000"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559139    2557 flags.go:64] FLAG: --max-pods="110"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559142    2557 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559145    2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559148    2557 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559151    2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559154    2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559157    2557 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559160    2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559167    2557 flags.go:64] FLAG: --node-status-max-images="50"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559170    2557 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559173    2557 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 06 10:29:46.566379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559176    2557 flags.go:64] FLAG: --pod-cidr=""
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559179    2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559184    2557 flags.go:64] FLAG: --pod-manifest-path=""
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559187    2557 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559190    2557 flags.go:64] FLAG: --pods-per-core="0"
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559193    2557 flags.go:64] FLAG: --port="10250"
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559196    2557 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559199    2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f922e030b8c769b0"
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559202    2557 flags.go:64] FLAG: --qos-reserved=""
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559206    2557 flags.go:64] FLAG: --read-only-port="10255"
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559209    2557 flags.go:64] FLAG: --register-node="true"
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559212    2557 flags.go:64] FLAG: --register-with-taints=""
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559216    2557 flags.go:64] FLAG: --registry-burst="10"
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559224    2557 flags.go:64] FLAG: --registry-qps="5"
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559227    2557 flags.go:64] FLAG: --reserved-cpus=""
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559230    2557 flags.go:64] FLAG: --reserved-memory=""
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559240    2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559244    2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559248    2557 flags.go:64] FLAG: --rotate-certificates="false"
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559253    2557 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559256    2557 flags.go:64] FLAG: --runonce="false"
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559259    2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559262    2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559265    2557 flags.go:64] FLAG: --seccomp-default="false"
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559267    2557 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 06 10:29:46.567010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559271    2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559274    2557 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559277    2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559280    2557 flags.go:64] FLAG: --storage-driver-password="root"
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559283    2557 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559286    2557 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559289    2557 flags.go:64] FLAG: --storage-driver-user="root"
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559292    2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559295    2557 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559298    2557 flags.go:64] FLAG: --system-cgroups=""
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559301    2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559306    2557 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559309    2557 flags.go:64] FLAG: --tls-cert-file=""
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559312    2557 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559316    2557 flags.go:64] FLAG: --tls-min-version=""
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559319    2557 flags.go:64] FLAG: --tls-private-key-file=""
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559322    2557 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559325    2557 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559328    2557 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559331    2557 flags.go:64] FLAG: --v="2"
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559334    2557 flags.go:64] FLAG: --version="false"
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559339    2557 flags.go:64] FLAG: --vmodule=""
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559343    2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559346    2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559474    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 10:29:46.567624 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559479    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 10:29:46.568250 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559483    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 10:29:46.568250 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559486    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 10:29:46.568250 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559489    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 10:29:46.568250 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559491    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 10:29:46.568250 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559494    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 10:29:46.568250 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559497    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 10:29:46.568250 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559501    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 10:29:46.568250 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559505    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 10:29:46.568250 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559508    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 10:29:46.568250 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559511    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 10:29:46.568250 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559513    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 10:29:46.568250 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559516    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 10:29:46.568250 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559519    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 10:29:46.568250 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559521    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 10:29:46.568250 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559524    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 10:29:46.568250 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559527    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 10:29:46.568250 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559529    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 10:29:46.568250 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559532    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 10:29:46.568250 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559535    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559537    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559540    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559542    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559545    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559548    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559550    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559553    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559556    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559558    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559561    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559564    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559566    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559571    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559574    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559577    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559580    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559582    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559585    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559588    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 10:29:46.568745 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559590    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 10:29:46.569236 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559593    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 10:29:46.569236 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559595    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 10:29:46.569236 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559598    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 10:29:46.569236 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559601    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 10:29:46.569236 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559603    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 10:29:46.569236 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559606    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 10:29:46.569236 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559608    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 10:29:46.569236 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559611    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 10:29:46.569236 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559613    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 10:29:46.569236 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559616    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 10:29:46.569236 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559619    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 10:29:46.569236 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559621    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 10:29:46.569236 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559624    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 10:29:46.569236 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559628    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 10:29:46.569236 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559632    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 10:29:46.569236 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559635    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 10:29:46.569236 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559638    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 10:29:46.569236 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559640    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 10:29:46.569236 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559643    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559645    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559648    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559651    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559653    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559656    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559660    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559665    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559667    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559670    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559672    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559675    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559677    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559680    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559683    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559685    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559688    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559690    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559694    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559698    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 10:29:46.569746 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559701    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 10:29:46.570257 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559706    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 10:29:46.570257 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559710    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 10:29:46.570257 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559712    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 10:29:46.570257 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559715    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 10:29:46.570257 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559718    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 10:29:46.570257 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559721    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 10:29:46.570257 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559723    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 10:29:46.570257 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559726    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 10:29:46.570257 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559729    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 10:29:46.570257 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559731    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 10:29:46.570257 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559735    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 10:29:46.570257 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559739    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 10:29:46.570257 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559742    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 10:29:46.570257 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559745    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 10:29:46.570257 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559747    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 10:29:46.570257 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559750    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 10:29:46.570257 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559753    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 10:29:46.570257 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559757    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 10:29:46.570257 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.559762    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 10:29:46.570775 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.559768    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 06 10:29:46.570775 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.567323    2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 06 10:29:46.570775 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.567339    2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 06 10:29:46.570775 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.568251    2557 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 06 10:29:46.570775 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.568259    2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 06 10:29:46.570775 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568316    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 10:29:46.570775 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568321    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 10:29:46.570775 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568324    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 10:29:46.570775 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568327    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 10:29:46.570775 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568330    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 10:29:46.570775 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568332    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 10:29:46.570775 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568335    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 10:29:46.570775 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568338    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 10:29:46.570775 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568340    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 10:29:46.570775 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568343    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568345    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568348    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568350    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568353    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568356    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568358    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568361    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568363    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568366    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568368    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568371    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568373    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568376    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568379    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568381    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568384    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568387    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568389    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568392    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 10:29:46.571177 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568397    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568401    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568404    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568407    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568410    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568413    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568415    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568418    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568421    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568423    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568426    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568429    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568432    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568434    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568437    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568440    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568444    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568447    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568450    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568452    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 10:29:46.571684 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568467    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568471    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568473    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568476    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568479    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568482    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568485    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568487    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568489    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568492    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568494    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568497    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568500    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568504    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568507    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568509    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568512    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568515    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568518    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568520    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 10:29:46.572277 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568523    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 10:29:46.572776 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568525    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 10:29:46.572776 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568528    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 10:29:46.572776 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568530    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 10:29:46.572776 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568533    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 10:29:46.572776 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568535    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 10:29:46.572776 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568538    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 10:29:46.572776 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568541    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 10:29:46.572776 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568543    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 10:29:46.572776 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568545    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 10:29:46.572776 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568548    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 10:29:46.572776 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568551    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 10:29:46.572776 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568553    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 10:29:46.572776 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568556    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 10:29:46.572776 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568558    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 10:29:46.572776 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568561    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 10:29:46.572776 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568563    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 10:29:46.572776 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568566    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 10:29:46.572776 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568569    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 10:29:46.572776 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568571    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 10:29:46.573369 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568574    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 10:29:46.573369 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568576    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 10:29:46.573369 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568580    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 10:29:46.573369 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568584    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 10:29:46.573369 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568588    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 10:29:46.573369 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568591    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 10:29:46.573369 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568595    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 10:29:46.573369 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568597    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 10:29:46.573369 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568600    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 10:29:46.573369 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568602    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 10:29:46.573369 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.568607    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 06 10:29:46.573369 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568720    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 10:29:46.573369 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568724    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 10:29:46.573369 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568727    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 10:29:46.573369 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568730    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 10:29:46.573771 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568733    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 10:29:46.573771 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568736    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 10:29:46.573771 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568739    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 10:29:46.573771 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568742    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 10:29:46.573771 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568745    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 10:29:46.573771 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568747    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 10:29:46.573771 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568750    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 10:29:46.573771 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568753    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 10:29:46.573771 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568755    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 10:29:46.573771 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568758    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 10:29:46.573771 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568760    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 10:29:46.573771 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568763    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 10:29:46.573771 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568766    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 10:29:46.573771 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568769    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 10:29:46.573771 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568771    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 10:29:46.573771 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568774    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 10:29:46.573771 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568776    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 10:29:46.573771 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568778    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 10:29:46.573771 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568781    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568783    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568786    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568789    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568792    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568795    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568798    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568801    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568804    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568807    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568809    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568812    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568814    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568817    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568819    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568822    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568824    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568827    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568829    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568832    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 10:29:46.574292 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568834    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568837    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568839    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568842    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568844    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568847    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568849    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568851    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568854    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568857    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568860    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568864    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568866    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568869    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568871    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568874    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568877    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568880    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568882    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568887    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 10:29:46.575218 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568890    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 10:29:46.576083 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568894    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 10:29:46.576083 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568897    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 10:29:46.576083 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568899    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 10:29:46.576083 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568902    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 10:29:46.576083 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568905    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 10:29:46.576083 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568908    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 10:29:46.576083 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568911    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 10:29:46.576083 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568913    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 10:29:46.576083 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568916    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 10:29:46.576083 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568919    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 10:29:46.576083 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568921    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 10:29:46.576083 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568924    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 10:29:46.576083 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568926    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 10:29:46.576083 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568929    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 10:29:46.576083 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568931    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 10:29:46.576083 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568934    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 10:29:46.576083 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568936    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 10:29:46.576083 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568939    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 10:29:46.576083 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568942    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 10:29:46.576832 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568944    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 10:29:46.576832 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568946    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 10:29:46.576832 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568949    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 10:29:46.576832 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568953    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 10:29:46.576832 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568956    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 10:29:46.576832 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568959    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 10:29:46.576832 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568962    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 10:29:46.576832 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568965    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 10:29:46.576832 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568969    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 10:29:46.576832 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568971    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 10:29:46.576832 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568974    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 10:29:46.576832 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568976    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 10:29:46.576832 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568979    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 10:29:46.576832 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568982    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 10:29:46.576832 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568984    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 10:29:46.576832 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568987    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 10:29:46.576832 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:46.568989    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 10:29:46.577261 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.568994    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 06 10:29:46.577261 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.569100    2557 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 06 10:29:46.577261 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.570775    2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 06 10:29:46.577261 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.571883    2557 server.go:1019] "Starting client certificate rotation"
Sep 06 10:29:46.577261 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.571996    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 06 10:29:46.577261 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.572049    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 06 10:29:46.577261 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.574723    2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 06 10:29:46.578850 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.578827    2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 06 10:29:46.590342 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.590320    2557 log.go:25] "Validated CRI v1 runtime API"
Sep 06 10:29:46.594326 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.594306    2557 log.go:25] "Validated CRI v1 image API"
Sep 06 10:29:46.595154 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.595136    2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 06 10:29:46.598675 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.598656    2557 fs.go:135] Filesystem UUIDs: map[0029914a-db44-431b-a262-87c224a8df15:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 e02fd572-03f5-42ab-a688-ab3ba6706879:/dev/nvme0n1p4]
Sep 06 10:29:46.598737 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.598676    2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 06 10:29:46.603821 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.603806    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 06 10:29:46.604502 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.604380    2557 manager.go:216] Machine: {Timestamp:2026-09-06 10:29:46.602936375 +0000 UTC m=+0.373748071 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3110582 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec213829518fa689275aec61389a54a9 SystemUUID:ec213829-518f-a689-275a-ec61389a54a9 BootID:d906151c-0f5e-4599-9ff6-052500090b9b Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c4:a1:60:46:57 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c4:a1:60:46:57 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:9e:04:d0:c4:b1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 06 10:29:46.604502 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.604500    2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 06 10:29:46.604596 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.604584    2557 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 06 10:29:46.608259 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.608231    2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 06 10:29:46.608425 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.608263    2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-153.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 06 10:29:46.609863 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.609851    2557 topology_manager.go:138] "Creating topology manager with none policy"
Sep 06 10:29:46.609922 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.609867    2557 container_manager_linux.go:309] "Creating device plugin manager"
Sep 06 10:29:46.609922 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.609880    2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 06 10:29:46.610692 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.610680    2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 06 10:29:46.611780 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.611768    2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 06 10:29:46.616357 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.616343    2557 state_mem.go:36] "Initialized new in-memory state store"
Sep 06 10:29:46.616503 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.616493    2557 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 06 10:29:46.616561 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.616529    2557 kubelet.go:486] "Attempting to sync node with API server"
Sep 06 10:29:46.616561 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.616544    2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 06 10:29:46.616561 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.616559    2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 06 10:29:46.616690 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.616572    2557 kubelet.go:398] "Adding apiserver pod source"
Sep 06 10:29:46.616690 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.616585    2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 06 10:29:46.616967 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.616954    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 06 10:29:46.617026 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.616977    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 06 10:29:46.619923 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.619801    2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 06 10:29:46.622071 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.622053    2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 06 10:29:46.622139 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.622090    2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 06 10:29:46.623655 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.623642    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 06 10:29:46.623718 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.623664    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 06 10:29:46.623718 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.623670    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 06 10:29:46.623718 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.623675    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 06 10:29:46.623718 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.623683    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 06 10:29:46.623718 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.623689    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 06 10:29:46.623718 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.623702    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 06 10:29:46.623718 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.623707    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 06 10:29:46.623718 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.623713    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 06 10:29:46.623718 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.623718    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 06 10:29:46.623975 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.623727    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 06 10:29:46.623975 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.623732    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 06 10:29:46.626036 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.626022    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 06 10:29:46.626073 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.626039    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 06 10:29:46.630101 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:46.630068    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 06 10:29:46.630189 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:46.630101    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-153.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 06 10:29:46.630189 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.630095    2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-153.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 06 10:29:46.630640 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.630620    2557 server.go:1268] "Started kubelet"
Sep 06 10:29:46.630744 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.630684    2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 06 10:29:46.630812 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.630758    2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 06 10:29:46.630854 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.630815    2557 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 06 10:29:46.631439 ip-10-0-133-153 systemd[1]: Started Kubernetes Kubelet.
Sep 06 10:29:46.632455 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.632441    2557 server.go:310] "Adding debug handlers to kubelet server"
Sep 06 10:29:46.632562 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.632494    2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 06 10:29:46.638238 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:46.638215    2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 06 10:29:46.638626 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.638609    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 06 10:29:46.639138 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.639115    2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 06 10:29:46.639249 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:46.639231    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-153.ec2.internal\" not found"
Sep 06 10:29:46.639709 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:46.638175    2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-153.ec2.internal.18d2b49c95339589  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-153.ec2.internal,UID:ip-10-0-133-153.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-153.ec2.internal,},FirstTimestamp:2026-09-06 10:29:46.630591881 +0000 UTC m=+0.401403577,LastTimestamp:2026-09-06 10:29:46.630591881 +0000 UTC m=+0.401403577,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-153.ec2.internal,}"
Sep 06 10:29:46.640116 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.639960    2557 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 06 10:29:46.640116 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.640117    2557 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 06 10:29:46.640260 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.639977    2557 factory.go:55] Registering systemd factory
Sep 06 10:29:46.640260 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.639957    2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 06 10:29:46.640260 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.640152    2557 factory.go:223] Registration of the systemd container factory successfully
Sep 06 10:29:46.640260 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.640211    2557 reconstruct.go:97] "Volume reconstruction finished"
Sep 06 10:29:46.640260 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.640221    2557 reconciler.go:29] "Reconciler: start to sync state"
Sep 06 10:29:46.640422 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.640412    2557 factory.go:153] Registering CRI-O factory
Sep 06 10:29:46.640451 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.640424    2557 factory.go:223] Registration of the crio container factory successfully
Sep 06 10:29:46.640503 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.640491    2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 06 10:29:46.640538 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.640509    2557 factory.go:103] Registering Raw factory
Sep 06 10:29:46.640538 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.640523    2557 manager.go:1196] Started watching for new ooms in manager
Sep 06 10:29:46.640953 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.640941    2557 manager.go:349] Starting recovery of all containers
Sep 06 10:29:46.647032 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:46.647004    2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-153.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 06 10:29:46.647032 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:46.647006    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 06 10:29:46.651586 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.651559    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jr6wz"
Sep 06 10:29:46.653595 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.653573    2557 manager.go:354] Recovery completed
Sep 06 10:29:46.655892 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:46.655872    2557 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 06 10:29:46.658832 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.658813    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jr6wz"
Sep 06 10:29:46.658918 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.658898    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 10:29:46.661219 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.661198    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 10:29:46.661287 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.661236    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 10:29:46.661287 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.661251    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasSufficientPID"
Sep 06 10:29:46.661822 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.661808    2557 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 06 10:29:46.661822 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.661822    2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 06 10:29:46.661918 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.661836    2557 state_mem.go:36] "Initialized new in-memory state store"
Sep 06 10:29:46.661918 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.661814    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 10:29:46.662609 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.662593    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 10:29:46.662692 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.662625    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 10:29:46.662692 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.662641    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasSufficientPID"
Sep 06 10:29:46.663146 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:46.663078    2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-153.ec2.internal.18d2b49c9706e891  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-153.ec2.internal,UID:ip-10-0-133-153.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-153.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-153.ec2.internal,},FirstTimestamp:2026-09-06 10:29:46.661218449 +0000 UTC m=+0.432030149,LastTimestamp:2026-09-06 10:29:46.661218449 +0000 UTC m=+0.432030149,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-153.ec2.internal,}"
Sep 06 10:29:46.664039 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.664026    2557 policy_none.go:49] "None policy: Start"
Sep 06 10:29:46.664104 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.664044    2557 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 06 10:29:46.664104 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.664065    2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 06 10:29:46.666187 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.666174    2557 policy_none.go:47] "Start"
Sep 06 10:29:46.696039 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.696024    2557 manager.go:337] "Starting Device Plugin manager"
Sep 06 10:29:46.718542 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:46.696070    2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 06 10:29:46.718542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.696080    2557 server.go:87] "Starting device plugin registration server"
Sep 06 10:29:46.718542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.696444    2557 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 06 10:29:46.718542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.696469    2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 06 10:29:46.718542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.696588    2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 06 10:29:46.718542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.696719    2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 06 10:29:46.718542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.696729    2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 06 10:29:46.718542 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:46.698766    2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 06 10:29:46.718542 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:46.698824    2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-153.ec2.internal\" not found"
Sep 06 10:29:46.718542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.707726    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 06 10:29:46.718542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.708840    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 06 10:29:46.718542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.708882    2557 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 06 10:29:46.718542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.708901    2557 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 06 10:29:46.718542 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:46.708942    2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 06 10:29:46.718542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.710705    2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 10:29:46.796903 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.796873    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 10:29:46.798021 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.797945    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 10:29:46.798021 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.797976    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 10:29:46.798021 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.797987    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasSufficientPID"
Sep 06 10:29:46.798021 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.798009    2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-153.ec2.internal"
Sep 06 10:29:46.807170 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.807151    2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-153.ec2.internal"
Sep 06 10:29:46.807170 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:46.807176    2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-153.ec2.internal\": node \"ip-10-0-133-153.ec2.internal\" not found"
Sep 06 10:29:46.809392 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.809374    2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-153.ec2.internal"]
Sep 06 10:29:46.809477 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.809435    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 10:29:46.811131 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.811116    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 10:29:46.811200 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.811142    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 10:29:46.811200 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.811153    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasSufficientPID"
Sep 06 10:29:46.812973 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.812961    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 10:29:46.813093 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.813076    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal"
Sep 06 10:29:46.813127 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.813106    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 10:29:46.813636 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.813620    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 10:29:46.813636 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.813630    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 10:29:46.813713 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.813647    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 10:29:46.813713 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.813657    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasSufficientPID"
Sep 06 10:29:46.813713 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.813671    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 10:29:46.813713 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.813683    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasSufficientPID"
Sep 06 10:29:46.815417 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.815403    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-153.ec2.internal"
Sep 06 10:29:46.815485 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.815426    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 10:29:46.816077 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.816064    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 10:29:46.816135 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.816085    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 10:29:46.816135 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.816098    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeHasSufficientPID"
Sep 06 10:29:46.830000 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:46.829982    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-153.ec2.internal\" not found" node="ip-10-0-133-153.ec2.internal"
Sep 06 10:29:46.834192 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:46.834178    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-153.ec2.internal\" not found" node="ip-10-0-133-153.ec2.internal"
Sep 06 10:29:46.837556 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:46.837542    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-153.ec2.internal\" not found"
Sep 06 10:29:46.841627 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.841607    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9303e4c798e32985c783de3b6c32d048-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal\" (UID: \"9303e4c798e32985c783de3b6c32d048\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal"
Sep 06 10:29:46.841711 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.841636    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9303e4c798e32985c783de3b6c32d048-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal\" (UID: \"9303e4c798e32985c783de3b6c32d048\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal"
Sep 06 10:29:46.841711 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.841653    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b6ddc99b2f4bb71ea7246abe1dc030e6-config\") pod \"kube-apiserver-proxy-ip-10-0-133-153.ec2.internal\" (UID: \"b6ddc99b2f4bb71ea7246abe1dc030e6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-153.ec2.internal"
Sep 06 10:29:46.937642 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:46.937618    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-153.ec2.internal\" not found"
Sep 06 10:29:46.941968 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.941951    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9303e4c798e32985c783de3b6c32d048-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal\" (UID: \"9303e4c798e32985c783de3b6c32d048\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal"
Sep 06 10:29:46.942030 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.941978    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9303e4c798e32985c783de3b6c32d048-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal\" (UID: \"9303e4c798e32985c783de3b6c32d048\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal"
Sep 06 10:29:46.942030 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.941993    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b6ddc99b2f4bb71ea7246abe1dc030e6-config\") pod \"kube-apiserver-proxy-ip-10-0-133-153.ec2.internal\" (UID: \"b6ddc99b2f4bb71ea7246abe1dc030e6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-153.ec2.internal"
Sep 06 10:29:46.942131 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.942045    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9303e4c798e32985c783de3b6c32d048-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal\" (UID: \"9303e4c798e32985c783de3b6c32d048\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal"
Sep 06 10:29:46.942131 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.942070    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b6ddc99b2f4bb71ea7246abe1dc030e6-config\") pod \"kube-apiserver-proxy-ip-10-0-133-153.ec2.internal\" (UID: \"b6ddc99b2f4bb71ea7246abe1dc030e6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-153.ec2.internal"
Sep 06 10:29:46.942131 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:46.942118    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9303e4c798e32985c783de3b6c32d048-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal\" (UID: \"9303e4c798e32985c783de3b6c32d048\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal"
Sep 06 10:29:47.038153 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:47.038110    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-153.ec2.internal\" not found"
Sep 06 10:29:47.131904 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:47.131831    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal"
Sep 06 10:29:47.138787 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:47.138767    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-153.ec2.internal\" not found"
Sep 06 10:29:47.138859 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:47.138810    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-153.ec2.internal"
Sep 06 10:29:47.239791 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:47.239753    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-153.ec2.internal\" not found"
Sep 06 10:29:47.340192 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:47.340155    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-153.ec2.internal\" not found"
Sep 06 10:29:47.440666 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:47.440592    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-153.ec2.internal\" not found"
Sep 06 10:29:47.453022 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:47.453001    2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 10:29:48.861878 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:48.861853    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-153.ec2.internal\" not found"
Sep 06 10:29:48.893163 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:48.893147    2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 06 10:29:48.893522 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:48.893250    2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 06 10:29:48.893522 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:48.893291    2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 06 10:29:48.962001 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:48.961979    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-153.ec2.internal\" not found"
Sep 06 10:29:48.986998 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:48.986968    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-05 10:24:47 +0000 UTC" deadline="2028-05-04 02:27:50.236672031 +0000 UTC"
Sep 06 10:29:48.986998 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:48.986992    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14535h58m1.249682438s"
Sep 06 10:29:49.062126 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:49.062092    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-153.ec2.internal\" not found"
Sep 06 10:29:49.162535 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:49.162513    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-153.ec2.internal\" not found"
Sep 06 10:29:49.169470 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:49.169434    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6ddc99b2f4bb71ea7246abe1dc030e6.slice/crio-646a45ee2be93dbfd8a7cf609d461a477a6cf862810fc0c47904efdaf3e01d6a WatchSource:0}: Error finding container 646a45ee2be93dbfd8a7cf609d461a477a6cf862810fc0c47904efdaf3e01d6a: Status 404 returned error can't find the container with id 646a45ee2be93dbfd8a7cf609d461a477a6cf862810fc0c47904efdaf3e01d6a
Sep 06 10:29:49.169802 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:49.169786    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9303e4c798e32985c783de3b6c32d048.slice/crio-e9d1427845b92330bb74c447b095968186d866e73e517e556ef520b3f775d8fe WatchSource:0}: Error finding container e9d1427845b92330bb74c447b095968186d866e73e517e556ef520b3f775d8fe: Status 404 returned error can't find the container with id e9d1427845b92330bb74c447b095968186d866e73e517e556ef520b3f775d8fe
Sep 06 10:29:49.176141 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.176126    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 10:29:49.263574 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:49.263549    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-153.ec2.internal\" not found"
Sep 06 10:29:49.331827 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.331785    2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 10:29:49.337572 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.337553    2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 10:29:49.360515 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.360478    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal"
Sep 06 10:29:49.373731 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.373714    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 06 10:29:49.374759 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.374747    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-153.ec2.internal"
Sep 06 10:29:49.383144 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.383133    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 06 10:29:49.885651 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.885625    2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 10:29:49.938795 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.938656    2557 apiserver.go:52] "Watching apiserver"
Sep 06 10:29:49.946924 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.946514    2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 06 10:29:49.948099 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.948072    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-hbgj5","kube-system/konnectivity-agent-kpqgt","kube-system/kube-apiserver-proxy-ip-10-0-133-153.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h","openshift-cluster-node-tuning-operator/tuned-5kn44","openshift-dns/node-resolver-t57dx","openshift-image-registry/node-ca-st44k","openshift-multus/multus-additional-cni-plugins-l57sp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal","openshift-multus/multus-z72zd","openshift-multus/network-metrics-daemon-pgxvx","openshift-network-diagnostics/network-check-target-bcjj6","openshift-network-operator/iptables-alerter-789d7","openshift-ovn-kubernetes/ovnkube-node-tw474"]
Sep 06 10:29:49.951173 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.950576    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-st44k"
Sep 06 10:29:49.953353 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.952856    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-789d7"
Sep 06 10:29:49.953815 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.953584    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 06 10:29:49.953815 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.953604    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 06 10:29:49.953815 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.953642    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-v2mjb\""
Sep 06 10:29:49.953815 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.953801    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 06 10:29:49.955383 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.955060    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 06 10:29:49.955383 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.955338    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.955608 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.955469    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rc7zw\""
Sep 06 10:29:49.955750 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.955732    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 06 10:29:49.955834 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.955816    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 06 10:29:49.957624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.957376    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:49.957955 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.957935    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 06 10:29:49.958829 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.958810    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 06 10:29:49.958975 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.958847    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 06 10:29:49.959068 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.959049    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 06 10:29:49.959844 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.959825    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 06 10:29:49.960509 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.960230    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5gptd\""
Sep 06 10:29:49.960509 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.960358    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 06 10:29:49.960658 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.960585    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bwn8j\""
Sep 06 10:29:49.960658 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.960618    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 06 10:29:49.960812 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.960754    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 06 10:29:49.960865 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.960856    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 06 10:29:49.961015 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.961001    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 06 10:29:49.962304 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.962286    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:49.964296 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.964275    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 06 10:29:49.964646 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.964622    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 06 10:29:49.964845 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.964826    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zhgbb\""
Sep 06 10:29:49.965687 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.965666    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t57dx"
Sep 06 10:29:49.967940 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.967918    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:49.968474 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.968454    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 06 10:29:49.968578 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.968525    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 06 10:29:49.968635 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.968462    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g722g\""
Sep 06 10:29:49.970331 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.970307    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 06 10:29:49.970449 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.970430    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 06 10:29:49.970524 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.970459    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 06 10:29:49.970861 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.970839    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 06 10:29:49.971516 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.971473    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-log-socket\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.971602 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.971546    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vcc7\" (UniqueName: \"kubernetes.io/projected/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-kube-api-access-5vcc7\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.971602 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.971584    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nw4c\" (UniqueName: \"kubernetes.io/projected/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-kube-api-access-8nw4c\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:49.971706 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.971618    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-etc-sysctl-conf\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:49.971706 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.971649    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-sys\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:49.971851 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.971823    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e037c2c9-f192-4742-b594-adecc8c04606-etc-tuned\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:49.971938 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.971867    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3-serviceca\") pod \"node-ca-st44k\" (UID: \"d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3\") " pod="openshift-image-registry/node-ca-st44k"
Sep 06 10:29:49.971938 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.971898    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-kubelet\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.971938 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.971928    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-systemd-units\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.972105 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.971993    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-run-openvswitch\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.972105 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.972024    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-cni-bin\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.972105 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.972049    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-ovn-node-metrics-cert\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.972105 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.972079    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-etc-modprobe-d\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:49.972285 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.972111    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-node-log\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.972285 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.972144    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-run-ovn-kubernetes\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.972285 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.972170    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-etc-sysconfig\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:49.972285 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.972207    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-etc-sysctl-d\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:49.972516 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.972379    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-etc-systemd\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:49.972516 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.972410    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-run\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:49.972516 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.972454    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-lib-modules\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:49.972516 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.972501    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8aef3159-3c8b-434c-91d6-c50e26c4c6ec-host-slash\") pod \"iptables-alerter-789d7\" (UID: \"8aef3159-3c8b-434c-91d6-c50e26c4c6ec\") " pod="openshift-network-operator/iptables-alerter-789d7"
Sep 06 10:29:49.972745 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.972529    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfmzl\" (UniqueName: \"kubernetes.io/projected/8aef3159-3c8b-434c-91d6-c50e26c4c6ec-kube-api-access-lfmzl\") pod \"iptables-alerter-789d7\" (UID: \"8aef3159-3c8b-434c-91d6-c50e26c4c6ec\") " pod="openshift-network-operator/iptables-alerter-789d7"
Sep 06 10:29:49.972745 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.972561    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-ovnkube-config\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.972745 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.972590    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-ovnkube-script-lib\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.972745 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.972635    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:49.972745 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.972662    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3-host\") pod \"node-ca-st44k\" (UID: \"d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3\") " pod="openshift-image-registry/node-ca-st44k"
Sep 06 10:29:49.972745 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.972687    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-run-netns\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.972745 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.972715    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-cni-netd\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.973136 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.971502    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:29:49.973136 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:49.972852    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hbgj5" podUID="503a2587-73cc-43a0-b058-242b6b7ec81b"
Sep 06 10:29:49.973136 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.971550    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8r82j\""
Sep 06 10:29:49.973373 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.972751    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.973446 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.973387    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-socket-dir\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:49.973446 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.973410    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-var-lib-kubelet\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:49.973446 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.973427    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e037c2c9-f192-4742-b594-adecc8c04606-tmp\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:49.973649 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.973447    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk9v2\" (UniqueName: \"kubernetes.io/projected/e037c2c9-f192-4742-b594-adecc8c04606-kube-api-access-hk9v2\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:49.973649 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.973476    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-slash\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.973649 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.973534    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-run-ovn\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.973649 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.973600    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 06 10:29:49.973848 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.973675    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-registration-dir\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:49.973848 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.973705    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-etc-selinux\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:49.973940 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.973858    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-sys-fs\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:49.973940 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.973886    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-host\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:49.973940 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.973917    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlt6d\" (UniqueName: \"kubernetes.io/projected/d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3-kube-api-access-zlt6d\") pod \"node-ca-st44k\" (UID: \"d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3\") " pod="openshift-image-registry/node-ca-st44k"
Sep 06 10:29:49.974079 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.973975    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-etc-openvswitch\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.974079 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.974021    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-device-dir\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:49.974171 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.974077    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-etc-kubernetes\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:49.974171 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.974109    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8aef3159-3c8b-434c-91d6-c50e26c4c6ec-iptables-alerter-script\") pod \"iptables-alerter-789d7\" (UID: \"8aef3159-3c8b-434c-91d6-c50e26c4c6ec\") " pod="openshift-network-operator/iptables-alerter-789d7"
Sep 06 10:29:49.974171 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.974143    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-run-systemd\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.974318 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.974169    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 06 10:29:49.976219 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.974168    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-var-lib-openvswitch\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.976219 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.974504    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kpqgt"
Sep 06 10:29:49.976219 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.974501    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-env-overrides\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:49.976933 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.976914    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 06 10:29:49.977150 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.977134    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kf56d\""
Sep 06 10:29:49.977367 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.977323    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 06 10:29:49.977619 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.977603    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z72zd"
Sep 06 10:29:49.979641 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.979621    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 06 10:29:49.979912 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.979893    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bmjwh\""
Sep 06 10:29:49.980019 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.979897    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:29:49.980085 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:49.980068    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgxvx" podUID="5ef2cd14-0fae-417c-8d8a-b407e0c03903"
Sep 06 10:29:49.982444 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.982426    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:29:49.982551 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:49.982507    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcjj6" podUID="96763caf-341a-40a7-ba78-c94b6dedebfd"
Sep 06 10:29:49.996716 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:49.996695    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bqms4"
Sep 06 10:29:50.003032 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.003012    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bqms4"
Sep 06 10:29:50.035904 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.035864    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-153.ec2.internal" event={"ID":"b6ddc99b2f4bb71ea7246abe1dc030e6","Type":"ContainerStarted","Data":"646a45ee2be93dbfd8a7cf609d461a477a6cf862810fc0c47904efdaf3e01d6a"}
Sep 06 10:29:50.037127 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.037107    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal" event={"ID":"9303e4c798e32985c783de3b6c32d048","Type":"ContainerStarted","Data":"e9d1427845b92330bb74c447b095968186d866e73e517e556ef520b3f775d8fe"}
Sep 06 10:29:50.062881 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.062858    2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 06 10:29:50.075602 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.075547    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1fdbdca8-792e-4abe-8058-3cc6df71bedd-agent-certs\") pod \"konnectivity-agent-kpqgt\" (UID: \"1fdbdca8-792e-4abe-8058-3cc6df71bedd\") " pod="kube-system/konnectivity-agent-kpqgt"
Sep 06 10:29:50.075691 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.075614    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1fdbdca8-792e-4abe-8058-3cc6df71bedd-konnectivity-ca\") pod \"konnectivity-agent-kpqgt\" (UID: \"1fdbdca8-792e-4abe-8058-3cc6df71bedd\") " pod="kube-system/konnectivity-agent-kpqgt"
Sep 06 10:29:50.075691 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.075646    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3-serviceca\") pod \"node-ca-st44k\" (UID: \"d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3\") " pod="openshift-image-registry/node-ca-st44k"
Sep 06 10:29:50.075691 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.075671    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-systemd-units\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.075854 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.075708    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-run-openvswitch\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.075854 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.075730    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-cni-bin\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.075854 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.075760    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-ovn-node-metrics-cert\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.075854 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.075782    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-etc-modprobe-d\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.075854 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.075806    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-host-var-lib-kubelet\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.075854 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.075826    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-run-ovn-kubernetes\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.075854 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.075847    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-lib-modules\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.076204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.075884    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.076204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.075909    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4mw7\" (UniqueName: \"kubernetes.io/projected/8a42dde3-cdd1-49a0-86f1-c67b65ca9cee-kube-api-access-s4mw7\") pod \"node-resolver-t57dx\" (UID: \"8a42dde3-cdd1-49a0-86f1-c67b65ca9cee\") " pod="openshift-dns/node-resolver-t57dx"
Sep 06 10:29:50.076204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.075924    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-ovnkube-config\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.076204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.075946    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-ovnkube-script-lib\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.076204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.075964    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-multus-daemon-config\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.076204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.075985    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngkp5\" (UniqueName: \"kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5\") pod \"network-check-target-bcjj6\" (UID: \"96763caf-341a-40a7-ba78-c94b6dedebfd\") " pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:29:50.076204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076000    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.076204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076015    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret\") pod \"global-pull-secret-syncer-hbgj5\" (UID: \"503a2587-73cc-43a0-b058-242b6b7ec81b\") " pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:29:50.076204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076044    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.076204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076062    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-socket-dir\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:50.076204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076079    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-var-lib-kubelet\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.076204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076093    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e037c2c9-f192-4742-b594-adecc8c04606-tmp\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.076204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076106    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3-serviceca\") pod \"node-ca-st44k\" (UID: \"d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3\") " pod="openshift-image-registry/node-ca-st44k"
Sep 06 10:29:50.076204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076118    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-run-ovn\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.076204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076140    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-etc-selinux\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:50.076204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076163    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7xb9\" (UniqueName: \"kubernetes.io/projected/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-kube-api-access-f7xb9\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.076972 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076198    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlt6d\" (UniqueName: \"kubernetes.io/projected/d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3-kube-api-access-zlt6d\") pod \"node-ca-st44k\" (UID: \"d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3\") " pod="openshift-image-registry/node-ca-st44k"
Sep 06 10:29:50.076972 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076213    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-device-dir\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:50.076972 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076227    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs\") pod \"network-metrics-daemon-pgxvx\" (UID: \"5ef2cd14-0fae-417c-8d8a-b407e0c03903\") " pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:29:50.076972 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076252    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/503a2587-73cc-43a0-b058-242b6b7ec81b-dbus\") pod \"global-pull-secret-syncer-hbgj5\" (UID: \"503a2587-73cc-43a0-b058-242b6b7ec81b\") " pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:29:50.076972 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076280    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8aef3159-3c8b-434c-91d6-c50e26c4c6ec-iptables-alerter-script\") pod \"iptables-alerter-789d7\" (UID: \"8aef3159-3c8b-434c-91d6-c50e26c4c6ec\") " pod="openshift-network-operator/iptables-alerter-789d7"
Sep 06 10:29:50.076972 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076298    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-cni-binary-copy\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.076972 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076318    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-log-socket\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.076972 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076338    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vcc7\" (UniqueName: \"kubernetes.io/projected/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-kube-api-access-5vcc7\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.076972 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076359    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-var-lib-kubelet\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.076972 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076368    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nw4c\" (UniqueName: \"kubernetes.io/projected/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-kube-api-access-8nw4c\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:50.076972 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076420    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-etc-sysctl-conf\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.076972 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076445    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-sys\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.076972 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076472    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-hostroot\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.076972 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076516    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-etc-kubernetes\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.076972 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076541    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-kubelet\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.076972 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076552    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-run-ovn\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.076972 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076621    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-etc-selinux\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:50.077699 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076644    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-run\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.077699 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076671    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-host-run-netns\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.077699 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076695    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-device-dir\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:50.077699 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076716    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-host-var-lib-cni-bin\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.077699 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076728    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-ovnkube-config\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.077699 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076744    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-host-run-multus-certs\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.077699 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076769    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-cni-binary-copy\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.077699 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076804    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.077699 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076809    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-systemd-units\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.077699 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076828    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-node-log\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.077699 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076862    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-run-openvswitch\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.077699 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076875    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-etc-sysconfig\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.077699 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076901    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-etc-sysctl-d\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.077699 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076908    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-cni-bin\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.077699 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076926    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-etc-systemd\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.077699 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.076974    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-multus-conf-dir\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.077699 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077004    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89hxn\" (UniqueName: \"kubernetes.io/projected/5ef2cd14-0fae-417c-8d8a-b407e0c03903-kube-api-access-89hxn\") pod \"network-metrics-daemon-pgxvx\" (UID: \"5ef2cd14-0fae-417c-8d8a-b407e0c03903\") " pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:29:50.078478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077036    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8aef3159-3c8b-434c-91d6-c50e26c4c6ec-host-slash\") pod \"iptables-alerter-789d7\" (UID: \"8aef3159-3c8b-434c-91d6-c50e26c4c6ec\") " pod="openshift-network-operator/iptables-alerter-789d7"
Sep 06 10:29:50.078478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077098    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfmzl\" (UniqueName: \"kubernetes.io/projected/8aef3159-3c8b-434c-91d6-c50e26c4c6ec-kube-api-access-lfmzl\") pod \"iptables-alerter-789d7\" (UID: \"8aef3159-3c8b-434c-91d6-c50e26c4c6ec\") " pod="openshift-network-operator/iptables-alerter-789d7"
Sep 06 10:29:50.078478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077137    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:50.078478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077208    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-system-cni-dir\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.078478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077221    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-etc-modprobe-d\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.078478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077275    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8aef3159-3c8b-434c-91d6-c50e26c4c6ec-iptables-alerter-script\") pod \"iptables-alerter-789d7\" (UID: \"8aef3159-3c8b-434c-91d6-c50e26c4c6ec\") " pod="openshift-network-operator/iptables-alerter-789d7"
Sep 06 10:29:50.078478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077264    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8a42dde3-cdd1-49a0-86f1-c67b65ca9cee-hosts-file\") pod \"node-resolver-t57dx\" (UID: \"8a42dde3-cdd1-49a0-86f1-c67b65ca9cee\") " pod="openshift-dns/node-resolver-t57dx"
Sep 06 10:29:50.078478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077299    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-run-ovn-kubernetes\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.078478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077434    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a42dde3-cdd1-49a0-86f1-c67b65ca9cee-tmp-dir\") pod \"node-resolver-t57dx\" (UID: \"8a42dde3-cdd1-49a0-86f1-c67b65ca9cee\") " pod="openshift-dns/node-resolver-t57dx"
Sep 06 10:29:50.078478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077435    2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 06 10:29:50.078478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077462    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-lib-modules\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.078478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077464    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-multus-cni-dir\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.078478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077523    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-log-socket\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.078478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077537    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3-host\") pod \"node-ca-st44k\" (UID: \"d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3\") " pod="openshift-image-registry/node-ca-st44k"
Sep 06 10:29:50.078478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077594    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-run-netns\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.078478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077628    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-cni-netd\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.078478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077635    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3-host\") pod \"node-ca-st44k\" (UID: \"d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3\") " pod="openshift-image-registry/node-ca-st44k"
Sep 06 10:29:50.078478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077658    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk9v2\" (UniqueName: \"kubernetes.io/projected/e037c2c9-f192-4742-b594-adecc8c04606-kube-api-access-hk9v2\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.079290 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077692    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/503a2587-73cc-43a0-b058-242b6b7ec81b-kubelet-config\") pod \"global-pull-secret-syncer-hbgj5\" (UID: \"503a2587-73cc-43a0-b058-242b6b7ec81b\") " pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:29:50.079290 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077718    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-node-log\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.079290 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077723    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-slash\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.079290 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077804    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.079290 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077820    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-etc-sysconfig\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.079290 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077913    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-etc-sysctl-d\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.079290 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.077921    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-socket-dir\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:50.079290 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.078918    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:50.079290 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.079035    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-etc-sysctl-conf\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.079290 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.079238    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-kubelet\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.079806 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.079313    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-run\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.079806 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.079436    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-run-netns\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.079806 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.079475    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-cni-netd\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.079806 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.079568    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-sys\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.079806 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.079615    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-registration-dir\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:50.079806 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.079647    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-sys-fs\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:50.079806 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.079680    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-host\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.079806 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.079724    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-etc-openvswitch\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.079806 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.079788    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-etc-kubernetes\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.080289 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.079817    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-multus-socket-dir-parent\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.080289 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.079850    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-host-var-lib-cni-multus\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.080289 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.079860    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-ovnkube-script-lib\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.080289 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.079882    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-run-systemd\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.080289 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.079934    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-etc-openvswitch\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.080289 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.080026    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-etc-kubernetes\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.080289 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.080145    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8aef3159-3c8b-434c-91d6-c50e26c4c6ec-host-slash\") pod \"iptables-alerter-789d7\" (UID: \"8aef3159-3c8b-434c-91d6-c50e26c4c6ec\") " pod="openshift-network-operator/iptables-alerter-789d7"
Sep 06 10:29:50.080289 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.080188    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-sys-fs\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:50.080289 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.080215    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-var-lib-openvswitch\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.080289 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.080245    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-etc-systemd\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.080289 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.080255    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-env-overrides\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.080812 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.080314    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e037c2c9-f192-4742-b594-adecc8c04606-host\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.080812 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.080319    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-run-systemd\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.080812 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.080353    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-system-cni-dir\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.080812 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.080387    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-cnibin\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.080812 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.080431    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e037c2c9-f192-4742-b594-adecc8c04606-etc-tuned\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.080812 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.080470    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-cnibin\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.080812 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.080526    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-host-run-k8s-cni-cncf-io\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.080812 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.080557    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t4jk\" (UniqueName: \"kubernetes.io/projected/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-kube-api-access-2t4jk\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.080812 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.080587    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-host-slash\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.080812 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.080696    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-var-lib-openvswitch\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.080812 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.080723    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-env-overrides\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.081345 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.080864    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-registration-dir\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:50.083834 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.083683    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-ovn-node-metrics-cert\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.084130 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.084082    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e037c2c9-f192-4742-b594-adecc8c04606-tmp\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.085323 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.085297    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e037c2c9-f192-4742-b594-adecc8c04606-etc-tuned\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.088168 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.088144    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vcc7\" (UniqueName: \"kubernetes.io/projected/cbcbc6f7-1879-4390-8b71-b2215b7fc9eb-kube-api-access-5vcc7\") pod \"ovnkube-node-tw474\" (UID: \"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.088582 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.088563    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlt6d\" (UniqueName: \"kubernetes.io/projected/d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3-kube-api-access-zlt6d\") pod \"node-ca-st44k\" (UID: \"d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3\") " pod="openshift-image-registry/node-ca-st44k"
Sep 06 10:29:50.088773 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.088753    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfmzl\" (UniqueName: \"kubernetes.io/projected/8aef3159-3c8b-434c-91d6-c50e26c4c6ec-kube-api-access-lfmzl\") pod \"iptables-alerter-789d7\" (UID: \"8aef3159-3c8b-434c-91d6-c50e26c4c6ec\") " pod="openshift-network-operator/iptables-alerter-789d7"
Sep 06 10:29:50.089255 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.089232    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk9v2\" (UniqueName: \"kubernetes.io/projected/e037c2c9-f192-4742-b594-adecc8c04606-kube-api-access-hk9v2\") pod \"tuned-5kn44\" (UID: \"e037c2c9-f192-4742-b594-adecc8c04606\") " pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.090226 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.090200    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nw4c\" (UniqueName: \"kubernetes.io/projected/0950615a-b7ad-4c9c-af8b-5adcf5c3c44e-kube-api-access-8nw4c\") pod \"aws-ebs-csi-driver-node-qsv7h\" (UID: \"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:50.181843 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.181775    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-host-var-lib-cni-multus\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.181843 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.181814    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-system-cni-dir\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.181843 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.181838    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-cnibin\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.182041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.181863    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-cnibin\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.182041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.181888    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-host-run-k8s-cni-cncf-io\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.182041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.181890    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-host-var-lib-cni-multus\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.182041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.181910    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t4jk\" (UniqueName: \"kubernetes.io/projected/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-kube-api-access-2t4jk\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.182041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.181951    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1fdbdca8-792e-4abe-8058-3cc6df71bedd-agent-certs\") pod \"konnectivity-agent-kpqgt\" (UID: \"1fdbdca8-792e-4abe-8058-3cc6df71bedd\") " pod="kube-system/konnectivity-agent-kpqgt"
Sep 06 10:29:50.182041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.181978    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1fdbdca8-792e-4abe-8058-3cc6df71bedd-konnectivity-ca\") pod \"konnectivity-agent-kpqgt\" (UID: \"1fdbdca8-792e-4abe-8058-3cc6df71bedd\") " pod="kube-system/konnectivity-agent-kpqgt"
Sep 06 10:29:50.182041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.181985    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-system-cni-dir\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.182041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182011    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-host-var-lib-kubelet\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.182041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.181951    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-cnibin\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.182530 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182060    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-host-var-lib-kubelet\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.182530 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182152    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-cnibin\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.182530 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182198    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-host-run-k8s-cni-cncf-io\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.182530 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182235    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.182530 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182268    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4mw7\" (UniqueName: \"kubernetes.io/projected/8a42dde3-cdd1-49a0-86f1-c67b65ca9cee-kube-api-access-s4mw7\") pod \"node-resolver-t57dx\" (UID: \"8a42dde3-cdd1-49a0-86f1-c67b65ca9cee\") " pod="openshift-dns/node-resolver-t57dx"
Sep 06 10:29:50.182530 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182294    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-multus-daemon-config\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.182814 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182555    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngkp5\" (UniqueName: \"kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5\") pod \"network-check-target-bcjj6\" (UID: \"96763caf-341a-40a7-ba78-c94b6dedebfd\") " pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:29:50.182814 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182585    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.182814 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182611    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret\") pod \"global-pull-secret-syncer-hbgj5\" (UID: \"503a2587-73cc-43a0-b058-242b6b7ec81b\") " pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:29:50.182814 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182608    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1fdbdca8-792e-4abe-8058-3cc6df71bedd-konnectivity-ca\") pod \"konnectivity-agent-kpqgt\" (UID: \"1fdbdca8-792e-4abe-8058-3cc6df71bedd\") " pod="kube-system/konnectivity-agent-kpqgt"
Sep 06 10:29:50.182814 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182644    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7xb9\" (UniqueName: \"kubernetes.io/projected/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-kube-api-access-f7xb9\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.182814 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182681    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs\") pod \"network-metrics-daemon-pgxvx\" (UID: \"5ef2cd14-0fae-417c-8d8a-b407e0c03903\") " pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:29:50.182814 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182704    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/503a2587-73cc-43a0-b058-242b6b7ec81b-dbus\") pod \"global-pull-secret-syncer-hbgj5\" (UID: \"503a2587-73cc-43a0-b058-242b6b7ec81b\") " pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:29:50.182814 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:50.182780    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 10:29:50.182814 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182802    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/503a2587-73cc-43a0-b058-242b6b7ec81b-dbus\") pod \"global-pull-secret-syncer-hbgj5\" (UID: \"503a2587-73cc-43a0-b058-242b6b7ec81b\") " pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:29:50.183262 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182822    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-cni-binary-copy\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.183262 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:50.182847    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs podName:5ef2cd14-0fae-417c-8d8a-b407e0c03903 nodeName:}" failed. No retries permitted until 2026-09-06 10:29:50.682832594 +0000 UTC m=+3.132613340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs") pod "network-metrics-daemon-pgxvx" (UID: "5ef2cd14-0fae-417c-8d8a-b407e0c03903") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 10:29:50.183262 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182857    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.183262 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182868    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-hostroot\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.183262 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182917    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-etc-kubernetes\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.183262 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182942    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-host-run-netns\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.183262 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182966    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-host-var-lib-cni-bin\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.183262 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.182990    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-host-run-multus-certs\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.183262 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183016    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-cni-binary-copy\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.183262 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183041    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.183262 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183068    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-multus-conf-dir\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.183262 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183091    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89hxn\" (UniqueName: \"kubernetes.io/projected/5ef2cd14-0fae-417c-8d8a-b407e0c03903-kube-api-access-89hxn\") pod \"network-metrics-daemon-pgxvx\" (UID: \"5ef2cd14-0fae-417c-8d8a-b407e0c03903\") " pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:29:50.183262 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183118    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-system-cni-dir\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.183262 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183141    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8a42dde3-cdd1-49a0-86f1-c67b65ca9cee-hosts-file\") pod \"node-resolver-t57dx\" (UID: \"8a42dde3-cdd1-49a0-86f1-c67b65ca9cee\") " pod="openshift-dns/node-resolver-t57dx"
Sep 06 10:29:50.183262 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183165    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a42dde3-cdd1-49a0-86f1-c67b65ca9cee-tmp-dir\") pod \"node-resolver-t57dx\" (UID: \"8a42dde3-cdd1-49a0-86f1-c67b65ca9cee\") " pod="openshift-dns/node-resolver-t57dx"
Sep 06 10:29:50.183262 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183187    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-multus-cni-dir\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.183262 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183230    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/503a2587-73cc-43a0-b058-242b6b7ec81b-kubelet-config\") pod \"global-pull-secret-syncer-hbgj5\" (UID: \"503a2587-73cc-43a0-b058-242b6b7ec81b\") " pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:29:50.184066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183262    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-multus-socket-dir-parent\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.184066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183334    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-multus-daemon-config\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.184066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183371    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-multus-socket-dir-parent\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.184066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183412    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-host-run-multus-certs\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.184066 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:50.183415    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 10:29:50.184066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183428    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-cni-binary-copy\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.184066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183459    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-system-cni-dir\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.184066 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:50.183474    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret podName:503a2587-73cc-43a0-b058-242b6b7ec81b nodeName:}" failed. No retries permitted until 2026-09-06 10:29:50.683459439 +0000 UTC m=+3.133240189 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret") pod "global-pull-secret-syncer-hbgj5" (UID: "503a2587-73cc-43a0-b058-242b6b7ec81b") : object "kube-system"/"original-pull-secret" not registered
Sep 06 10:29:50.184066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183478    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-hostroot\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.184066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183541    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8a42dde3-cdd1-49a0-86f1-c67b65ca9cee-hosts-file\") pod \"node-resolver-t57dx\" (UID: \"8a42dde3-cdd1-49a0-86f1-c67b65ca9cee\") " pod="openshift-dns/node-resolver-t57dx"
Sep 06 10:29:50.184066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183669    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.184066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183709    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-host-run-netns\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.184066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183721    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-host-var-lib-cni-bin\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.184066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183746    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-multus-conf-dir\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.184066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183762    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a42dde3-cdd1-49a0-86f1-c67b65ca9cee-tmp-dir\") pod \"node-resolver-t57dx\" (UID: \"8a42dde3-cdd1-49a0-86f1-c67b65ca9cee\") " pod="openshift-dns/node-resolver-t57dx"
Sep 06 10:29:50.184066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183784    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-etc-kubernetes\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.184066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183826    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-multus-cni-dir\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.184066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183834    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/503a2587-73cc-43a0-b058-242b6b7ec81b-kubelet-config\") pod \"global-pull-secret-syncer-hbgj5\" (UID: \"503a2587-73cc-43a0-b058-242b6b7ec81b\") " pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:29:50.184863 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.183906    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-cni-binary-copy\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.184863 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.184188    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.185196 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.185175    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1fdbdca8-792e-4abe-8058-3cc6df71bedd-agent-certs\") pod \"konnectivity-agent-kpqgt\" (UID: \"1fdbdca8-792e-4abe-8058-3cc6df71bedd\") " pod="kube-system/konnectivity-agent-kpqgt"
Sep 06 10:29:50.190725 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:50.190700    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 10:29:50.190725 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:50.190723    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 10:29:50.190878 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:50.190736    2557 projected.go:196] Error preparing data for projected volume kube-api-access-ngkp5 for pod openshift-network-diagnostics/network-check-target-bcjj6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 10:29:50.190878 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:50.190788    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5 podName:96763caf-341a-40a7-ba78-c94b6dedebfd nodeName:}" failed. No retries permitted until 2026-09-06 10:29:50.690774346 +0000 UTC m=+3.140555098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ngkp5" (UniqueName: "kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5") pod "network-check-target-bcjj6" (UID: "96763caf-341a-40a7-ba78-c94b6dedebfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 10:29:50.192919 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.192893    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4mw7\" (UniqueName: \"kubernetes.io/projected/8a42dde3-cdd1-49a0-86f1-c67b65ca9cee-kube-api-access-s4mw7\") pod \"node-resolver-t57dx\" (UID: \"8a42dde3-cdd1-49a0-86f1-c67b65ca9cee\") " pod="openshift-dns/node-resolver-t57dx"
Sep 06 10:29:50.193554 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.193533    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t4jk\" (UniqueName: \"kubernetes.io/projected/f27bdb46-dbcb-4639-ad28-3ed384e21ba6-kube-api-access-2t4jk\") pod \"multus-z72zd\" (UID: \"f27bdb46-dbcb-4639-ad28-3ed384e21ba6\") " pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.194014 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.193984    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7xb9\" (UniqueName: \"kubernetes.io/projected/b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3-kube-api-access-f7xb9\") pod \"multus-additional-cni-plugins-l57sp\" (UID: \"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3\") " pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.194087 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.194062    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89hxn\" (UniqueName: \"kubernetes.io/projected/5ef2cd14-0fae-417c-8d8a-b407e0c03903-kube-api-access-89hxn\") pod \"network-metrics-daemon-pgxvx\" (UID: \"5ef2cd14-0fae-417c-8d8a-b407e0c03903\") " pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:29:50.266259 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.266229    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-st44k"
Sep 06 10:29:50.279221 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.279197    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-789d7"
Sep 06 10:29:50.290069 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.290050    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:29:50.304874 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.304854    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h"
Sep 06 10:29:50.311751 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.311732    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5kn44"
Sep 06 10:29:50.321511 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.321453    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t57dx"
Sep 06 10:29:50.328173 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.328156    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l57sp"
Sep 06 10:29:50.336980 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.336958    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kpqgt"
Sep 06 10:29:50.343652 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.343632    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z72zd"
Sep 06 10:29:50.687663 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.687634    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret\") pod \"global-pull-secret-syncer-hbgj5\" (UID: \"503a2587-73cc-43a0-b058-242b6b7ec81b\") " pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:29:50.687846 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.687678    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs\") pod \"network-metrics-daemon-pgxvx\" (UID: \"5ef2cd14-0fae-417c-8d8a-b407e0c03903\") " pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:29:50.687846 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:50.687807    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 10:29:50.687960 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:50.687877    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret podName:503a2587-73cc-43a0-b058-242b6b7ec81b nodeName:}" failed. No retries permitted until 2026-09-06 10:29:51.68786017 +0000 UTC m=+4.137640934 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret") pod "global-pull-secret-syncer-hbgj5" (UID: "503a2587-73cc-43a0-b058-242b6b7ec81b") : object "kube-system"/"original-pull-secret" not registered
Sep 06 10:29:50.687960 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:50.687812    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 10:29:50.688061 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:50.687976    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs podName:5ef2cd14-0fae-417c-8d8a-b407e0c03903 nodeName:}" failed. No retries permitted until 2026-09-06 10:29:51.687945732 +0000 UTC m=+4.137726491 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs") pod "network-metrics-daemon-pgxvx" (UID: "5ef2cd14-0fae-417c-8d8a-b407e0c03903") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 10:29:50.789018 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.788969    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngkp5\" (UniqueName: \"kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5\") pod \"network-check-target-bcjj6\" (UID: \"96763caf-341a-40a7-ba78-c94b6dedebfd\") " pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:29:50.789197 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:50.789118    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 10:29:50.789197 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:50.789142    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 10:29:50.789197 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:50.789156    2557 projected.go:196] Error preparing data for projected volume kube-api-access-ngkp5 for pod openshift-network-diagnostics/network-check-target-bcjj6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 10:29:50.789359 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:50.789219    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5 podName:96763caf-341a-40a7-ba78-c94b6dedebfd nodeName:}" failed. No retries permitted until 2026-09-06 10:29:51.789201789 +0000 UTC m=+4.238982559 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ngkp5" (UniqueName: "kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5") pod "network-check-target-bcjj6" (UID: "96763caf-341a-40a7-ba78-c94b6dedebfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 10:29:50.859194 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:50.859171    2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 10:29:51.007036 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:51.006971    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-05 10:24:49 +0000 UTC" deadline="2028-04-14 23:00:30.95554108 +0000 UTC"
Sep 06 10:29:51.007036 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:51.006997    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14076h30m39.948547893s"
Sep 06 10:29:51.213885 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:51.213861    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode037c2c9_f192_4742_b594_adecc8c04606.slice/crio-f582b1688fe4bc431e67692158a259c147347759990966fe8fa56c7a4ad235f0 WatchSource:0}: Error finding container f582b1688fe4bc431e67692158a259c147347759990966fe8fa56c7a4ad235f0: Status 404 returned error can't find the container with id f582b1688fe4bc431e67692158a259c147347759990966fe8fa56c7a4ad235f0
Sep 06 10:29:51.215085 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:51.215010    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a9d66d_ae69_42c7_a978_1cdfdcc06fd3.slice/crio-800d83f9478ab3500e97b6f5335ffeb2634f53ed2d1d4bad1e03c9c107f28206 WatchSource:0}: Error finding container 800d83f9478ab3500e97b6f5335ffeb2634f53ed2d1d4bad1e03c9c107f28206: Status 404 returned error can't find the container with id 800d83f9478ab3500e97b6f5335ffeb2634f53ed2d1d4bad1e03c9c107f28206
Sep 06 10:29:51.219480 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:51.219458    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fdbdca8_792e_4abe_8058_3cc6df71bedd.slice/crio-8b6d805a5104d5d5dd4475e4b74e67e2a719f1ec45336a3b1cbfe02f033ee9b0 WatchSource:0}: Error finding container 8b6d805a5104d5d5dd4475e4b74e67e2a719f1ec45336a3b1cbfe02f033ee9b0: Status 404 returned error can't find the container with id 8b6d805a5104d5d5dd4475e4b74e67e2a719f1ec45336a3b1cbfe02f033ee9b0
Sep 06 10:29:51.220453 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:51.220434    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3d2bc45_9a8b_4efc_83ec_c5f57d5df6f3.slice/crio-05a526a1f4d4da1cd3376c8ca1ea0179a68b28d649f9b5ec3b40e5f4381acf52 WatchSource:0}: Error finding container 05a526a1f4d4da1cd3376c8ca1ea0179a68b28d649f9b5ec3b40e5f4381acf52: Status 404 returned error can't find the container with id 05a526a1f4d4da1cd3376c8ca1ea0179a68b28d649f9b5ec3b40e5f4381acf52
Sep 06 10:29:51.221592 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:51.221561    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0950615a_b7ad_4c9c_af8b_5adcf5c3c44e.slice/crio-4ae9395b6182053b86c997a19121db660b84e19e108cb5f123318fa09153865c WatchSource:0}: Error finding container 4ae9395b6182053b86c997a19121db660b84e19e108cb5f123318fa09153865c: Status 404 returned error can't find the container with id 4ae9395b6182053b86c997a19121db660b84e19e108cb5f123318fa09153865c
Sep 06 10:29:51.222076 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:51.222018    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aef3159_3c8b_434c_91d6_c50e26c4c6ec.slice/crio-ebd61c8c84e3354d2150718fca167c1f79877f78baea8a24b4588f852351e80b WatchSource:0}: Error finding container ebd61c8c84e3354d2150718fca167c1f79877f78baea8a24b4588f852351e80b: Status 404 returned error can't find the container with id ebd61c8c84e3354d2150718fca167c1f79877f78baea8a24b4588f852351e80b
Sep 06 10:29:51.223762 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:51.223507    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf27bdb46_dbcb_4639_ad28_3ed384e21ba6.slice/crio-6455490e2db6867e82b7af7d92f591b682ad4b4bde832dbb63f7c64b2b1efa8d WatchSource:0}: Error finding container 6455490e2db6867e82b7af7d92f591b682ad4b4bde832dbb63f7c64b2b1efa8d: Status 404 returned error can't find the container with id 6455490e2db6867e82b7af7d92f591b682ad4b4bde832dbb63f7c64b2b1efa8d
Sep 06 10:29:51.224849 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:51.224826    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbcbc6f7_1879_4390_8b71_b2215b7fc9eb.slice/crio-d6bab6371aed33cb85ca34998b1f7240ef141a247551434955121494a6926a1c WatchSource:0}: Error finding container d6bab6371aed33cb85ca34998b1f7240ef141a247551434955121494a6926a1c: Status 404 returned error can't find the container with id d6bab6371aed33cb85ca34998b1f7240ef141a247551434955121494a6926a1c
Sep 06 10:29:51.228547 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:29:51.228296    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a42dde3_cdd1_49a0_86f1_c67b65ca9cee.slice/crio-9f77456fe5ba1c703985c7246b35d8112a1edfe26ee69d0f4a6fc161a48974d9 WatchSource:0}: Error finding container 9f77456fe5ba1c703985c7246b35d8112a1edfe26ee69d0f4a6fc161a48974d9: Status 404 returned error can't find the container with id 9f77456fe5ba1c703985c7246b35d8112a1edfe26ee69d0f4a6fc161a48974d9
Sep 06 10:29:51.695053 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:51.695023    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret\") pod \"global-pull-secret-syncer-hbgj5\" (UID: \"503a2587-73cc-43a0-b058-242b6b7ec81b\") " pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:29:51.695198 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:51.695067    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs\") pod \"network-metrics-daemon-pgxvx\" (UID: \"5ef2cd14-0fae-417c-8d8a-b407e0c03903\") " pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:29:51.695198 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:51.695193    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 10:29:51.695317 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:51.695196    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 10:29:51.695317 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:51.695259    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs podName:5ef2cd14-0fae-417c-8d8a-b407e0c03903 nodeName:}" failed. No retries permitted until 2026-09-06 10:29:53.695239954 +0000 UTC m=+6.145020700 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs") pod "network-metrics-daemon-pgxvx" (UID: "5ef2cd14-0fae-417c-8d8a-b407e0c03903") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 10:29:51.695317 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:51.695278    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret podName:503a2587-73cc-43a0-b058-242b6b7ec81b nodeName:}" failed. No retries permitted until 2026-09-06 10:29:53.695269551 +0000 UTC m=+6.145050312 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret") pod "global-pull-secret-syncer-hbgj5" (UID: "503a2587-73cc-43a0-b058-242b6b7ec81b") : object "kube-system"/"original-pull-secret" not registered
Sep 06 10:29:51.796790 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:51.796319    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngkp5\" (UniqueName: \"kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5\") pod \"network-check-target-bcjj6\" (UID: \"96763caf-341a-40a7-ba78-c94b6dedebfd\") " pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:29:51.796790 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:51.796516    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 10:29:51.796790 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:51.796537    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 10:29:51.796790 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:51.796549    2557 projected.go:196] Error preparing data for projected volume kube-api-access-ngkp5 for pod openshift-network-diagnostics/network-check-target-bcjj6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 10:29:51.796790 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:51.796605    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5 podName:96763caf-341a-40a7-ba78-c94b6dedebfd nodeName:}" failed. No retries permitted until 2026-09-06 10:29:53.796586665 +0000 UTC m=+6.246367414 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ngkp5" (UniqueName: "kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5") pod "network-check-target-bcjj6" (UID: "96763caf-341a-40a7-ba78-c94b6dedebfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 10:29:52.007755 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:52.007690    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-05 10:24:49 +0000 UTC" deadline="2028-04-02 21:12:31.419159366 +0000 UTC"
Sep 06 10:29:52.007755 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:52.007754    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13786h42m39.411410206s"
Sep 06 10:29:52.034378 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:52.033684    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:29:52.034378 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:52.033813    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgxvx" podUID="5ef2cd14-0fae-417c-8d8a-b407e0c03903"
Sep 06 10:29:52.034378 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:52.034252    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:29:52.034378 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:52.034338    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcjj6" podUID="96763caf-341a-40a7-ba78-c94b6dedebfd"
Sep 06 10:29:52.038251 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:52.035784    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:29:52.038251 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:52.035877    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hbgj5" podUID="503a2587-73cc-43a0-b058-242b6b7ec81b"
Sep 06 10:29:52.049188 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:52.049125    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-153.ec2.internal" event={"ID":"b6ddc99b2f4bb71ea7246abe1dc030e6","Type":"ContainerStarted","Data":"fd43b96cf788b64d014bbe45f7c8caeb1954428de426c1e039cb3e1a078c4248"}
Sep 06 10:29:52.058954 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:52.058929    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l57sp" event={"ID":"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3","Type":"ContainerStarted","Data":"05a526a1f4d4da1cd3376c8ca1ea0179a68b28d649f9b5ec3b40e5f4381acf52"}
Sep 06 10:29:52.070975 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:52.070951    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tw474" event={"ID":"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb","Type":"ContainerStarted","Data":"d6bab6371aed33cb85ca34998b1f7240ef141a247551434955121494a6926a1c"}
Sep 06 10:29:52.079034 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:52.079009    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z72zd" event={"ID":"f27bdb46-dbcb-4639-ad28-3ed384e21ba6","Type":"ContainerStarted","Data":"6455490e2db6867e82b7af7d92f591b682ad4b4bde832dbb63f7c64b2b1efa8d"}
Sep 06 10:29:52.089117 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:52.089090    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-789d7" event={"ID":"8aef3159-3c8b-434c-91d6-c50e26c4c6ec","Type":"ContainerStarted","Data":"ebd61c8c84e3354d2150718fca167c1f79877f78baea8a24b4588f852351e80b"}
Sep 06 10:29:52.096257 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:52.096231    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h" event={"ID":"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e","Type":"ContainerStarted","Data":"4ae9395b6182053b86c997a19121db660b84e19e108cb5f123318fa09153865c"}
Sep 06 10:29:52.103096 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:52.103023    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kpqgt" event={"ID":"1fdbdca8-792e-4abe-8058-3cc6df71bedd","Type":"ContainerStarted","Data":"8b6d805a5104d5d5dd4475e4b74e67e2a719f1ec45336a3b1cbfe02f033ee9b0"}
Sep 06 10:29:52.111194 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:52.111160    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5kn44" event={"ID":"e037c2c9-f192-4742-b594-adecc8c04606","Type":"ContainerStarted","Data":"f582b1688fe4bc431e67692158a259c147347759990966fe8fa56c7a4ad235f0"}
Sep 06 10:29:52.114764 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:52.113942    2557 generic.go:356] "Generic (PLEG): container finished" podID="9303e4c798e32985c783de3b6c32d048" containerID="6519ac20638feab4d8859e959cb8ff9a3d601c5ae9130d68c8dbeabcaca73ebb" exitCode=0
Sep 06 10:29:52.114764 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:52.114012    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal" event={"ID":"9303e4c798e32985c783de3b6c32d048","Type":"ContainerDied","Data":"6519ac20638feab4d8859e959cb8ff9a3d601c5ae9130d68c8dbeabcaca73ebb"}
Sep 06 10:29:52.129212 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:52.129163    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-153.ec2.internal" podStartSLOduration=3.129146752 podStartE2EDuration="3.129146752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:29:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:29:52.066819059 +0000 UTC m=+4.516599829" watchObservedRunningTime="2026-09-06 10:29:52.129146752 +0000 UTC m=+4.578927527"
Sep 06 10:29:52.136794 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:52.136735    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t57dx" event={"ID":"8a42dde3-cdd1-49a0-86f1-c67b65ca9cee","Type":"ContainerStarted","Data":"9f77456fe5ba1c703985c7246b35d8112a1edfe26ee69d0f4a6fc161a48974d9"}
Sep 06 10:29:52.139293 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:52.139244    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-st44k" event={"ID":"d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3","Type":"ContainerStarted","Data":"800d83f9478ab3500e97b6f5335ffeb2634f53ed2d1d4bad1e03c9c107f28206"}
Sep 06 10:29:53.156921 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:53.156879    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal" event={"ID":"9303e4c798e32985c783de3b6c32d048","Type":"ContainerStarted","Data":"6766431cd99a772f435d79afdcc932a82e1502f534435bed214a69f805cf1cbf"}
Sep 06 10:29:53.724969 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:53.723949    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret\") pod \"global-pull-secret-syncer-hbgj5\" (UID: \"503a2587-73cc-43a0-b058-242b6b7ec81b\") " pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:29:53.724969 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:53.724095    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs\") pod \"network-metrics-daemon-pgxvx\" (UID: \"5ef2cd14-0fae-417c-8d8a-b407e0c03903\") " pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:29:53.724969 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:53.724226    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 10:29:53.724969 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:53.724292    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs podName:5ef2cd14-0fae-417c-8d8a-b407e0c03903 nodeName:}" failed. No retries permitted until 2026-09-06 10:29:57.72427144 +0000 UTC m=+10.174052200 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs") pod "network-metrics-daemon-pgxvx" (UID: "5ef2cd14-0fae-417c-8d8a-b407e0c03903") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 10:29:53.724969 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:53.724840    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 10:29:53.724969 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:53.724891    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret podName:503a2587-73cc-43a0-b058-242b6b7ec81b nodeName:}" failed. No retries permitted until 2026-09-06 10:29:57.724876174 +0000 UTC m=+10.174656937 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret") pod "global-pull-secret-syncer-hbgj5" (UID: "503a2587-73cc-43a0-b058-242b6b7ec81b") : object "kube-system"/"original-pull-secret" not registered
Sep 06 10:29:53.825438 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:53.825394    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngkp5\" (UniqueName: \"kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5\") pod \"network-check-target-bcjj6\" (UID: \"96763caf-341a-40a7-ba78-c94b6dedebfd\") " pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:29:53.825609 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:53.825574    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 10:29:53.825609 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:53.825596    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 10:29:53.825609 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:53.825606    2557 projected.go:196] Error preparing data for projected volume kube-api-access-ngkp5 for pod openshift-network-diagnostics/network-check-target-bcjj6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 10:29:53.825842 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:53.825662    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5 podName:96763caf-341a-40a7-ba78-c94b6dedebfd nodeName:}" failed. No retries permitted until 2026-09-06 10:29:57.825646477 +0000 UTC m=+10.275427249 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ngkp5" (UniqueName: "kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5") pod "network-check-target-bcjj6" (UID: "96763caf-341a-40a7-ba78-c94b6dedebfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 10:29:54.033336 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:54.033252    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:29:54.033524 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:54.033389    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgxvx" podUID="5ef2cd14-0fae-417c-8d8a-b407e0c03903"
Sep 06 10:29:54.033795 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:54.033775    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:29:54.033903 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:54.033869    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcjj6" podUID="96763caf-341a-40a7-ba78-c94b6dedebfd"
Sep 06 10:29:54.033968 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:54.033924    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:29:54.034053 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:54.033996    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hbgj5" podUID="503a2587-73cc-43a0-b058-242b6b7ec81b"
Sep 06 10:29:56.030393 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:56.030359    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:29:56.030889 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:56.030516    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcjj6" podUID="96763caf-341a-40a7-ba78-c94b6dedebfd"
Sep 06 10:29:56.030889 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:56.030359    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:29:56.031027 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:56.030978    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgxvx" podUID="5ef2cd14-0fae-417c-8d8a-b407e0c03903"
Sep 06 10:29:56.031087 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:56.031032    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:29:56.031136 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:56.031108    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hbgj5" podUID="503a2587-73cc-43a0-b058-242b6b7ec81b"
Sep 06 10:29:57.759069 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:57.758859    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret\") pod \"global-pull-secret-syncer-hbgj5\" (UID: \"503a2587-73cc-43a0-b058-242b6b7ec81b\") " pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:29:57.759544 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:57.759097    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs\") pod \"network-metrics-daemon-pgxvx\" (UID: \"5ef2cd14-0fae-417c-8d8a-b407e0c03903\") " pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:29:57.759544 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:57.759038    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 10:29:57.759544 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:57.759296    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret podName:503a2587-73cc-43a0-b058-242b6b7ec81b nodeName:}" failed. No retries permitted until 2026-09-06 10:30:05.75927588 +0000 UTC m=+18.209056645 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret") pod "global-pull-secret-syncer-hbgj5" (UID: "503a2587-73cc-43a0-b058-242b6b7ec81b") : object "kube-system"/"original-pull-secret" not registered
Sep 06 10:29:57.759728 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:57.759232    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 10:29:57.759728 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:57.759669    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs podName:5ef2cd14-0fae-417c-8d8a-b407e0c03903 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:05.759653006 +0000 UTC m=+18.209433772 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs") pod "network-metrics-daemon-pgxvx" (UID: "5ef2cd14-0fae-417c-8d8a-b407e0c03903") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 10:29:57.859684 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:57.859642    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngkp5\" (UniqueName: \"kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5\") pod \"network-check-target-bcjj6\" (UID: \"96763caf-341a-40a7-ba78-c94b6dedebfd\") " pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:29:57.859859 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:57.859843    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 10:29:57.859925 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:57.859865    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 10:29:57.859925 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:57.859878    2557 projected.go:196] Error preparing data for projected volume kube-api-access-ngkp5 for pod openshift-network-diagnostics/network-check-target-bcjj6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 10:29:57.860082 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:57.859931    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5 podName:96763caf-341a-40a7-ba78-c94b6dedebfd nodeName:}" failed. No retries permitted until 2026-09-06 10:30:05.859913091 +0000 UTC m=+18.309693860 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ngkp5" (UniqueName: "kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5") pod "network-check-target-bcjj6" (UID: "96763caf-341a-40a7-ba78-c94b6dedebfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 10:29:58.033978 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:58.033869    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:29:58.034211 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:58.034007    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:29:58.034397 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:58.034333    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hbgj5" podUID="503a2587-73cc-43a0-b058-242b6b7ec81b"
Sep 06 10:29:58.034397 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:29:58.034054    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:29:58.034596 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:58.034511    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgxvx" podUID="5ef2cd14-0fae-417c-8d8a-b407e0c03903"
Sep 06 10:29:58.034650 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:29:58.034604    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcjj6" podUID="96763caf-341a-40a7-ba78-c94b6dedebfd"
Sep 06 10:30:00.031245 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:00.030579    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:30:00.031245 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:00.030599    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:30:00.031245 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:00.030604    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:30:00.031245 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:00.030711    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hbgj5" podUID="503a2587-73cc-43a0-b058-242b6b7ec81b"
Sep 06 10:30:00.031245 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:00.031122    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgxvx" podUID="5ef2cd14-0fae-417c-8d8a-b407e0c03903"
Sep 06 10:30:00.031245 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:00.031210    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcjj6" podUID="96763caf-341a-40a7-ba78-c94b6dedebfd"
Sep 06 10:30:02.031114 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:02.031086    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:30:02.031114 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:02.031117    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:30:02.031624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:02.031087    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:30:02.031624 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:02.031218    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgxvx" podUID="5ef2cd14-0fae-417c-8d8a-b407e0c03903"
Sep 06 10:30:02.031624 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:02.031289    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hbgj5" podUID="503a2587-73cc-43a0-b058-242b6b7ec81b"
Sep 06 10:30:02.031624 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:02.031331    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcjj6" podUID="96763caf-341a-40a7-ba78-c94b6dedebfd"
Sep 06 10:30:04.030860 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:04.030826    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:30:04.031276 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:04.030833    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:30:04.031276 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:04.030934    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcjj6" podUID="96763caf-341a-40a7-ba78-c94b6dedebfd"
Sep 06 10:30:04.031276 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:04.030839    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:30:04.031276 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:04.031060    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hbgj5" podUID="503a2587-73cc-43a0-b058-242b6b7ec81b"
Sep 06 10:30:04.031276 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:04.031141    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgxvx" podUID="5ef2cd14-0fae-417c-8d8a-b407e0c03903"
Sep 06 10:30:05.818311 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:05.818269    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret\") pod \"global-pull-secret-syncer-hbgj5\" (UID: \"503a2587-73cc-43a0-b058-242b6b7ec81b\") " pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:30:05.818815 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:05.818323    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs\") pod \"network-metrics-daemon-pgxvx\" (UID: \"5ef2cd14-0fae-417c-8d8a-b407e0c03903\") " pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:30:05.818815 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:05.818410    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 10:30:05.818815 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:05.818433    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 10:30:05.818815 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:05.818483    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret podName:503a2587-73cc-43a0-b058-242b6b7ec81b nodeName:}" failed. No retries permitted until 2026-09-06 10:30:21.818463037 +0000 UTC m=+34.268243801 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret") pod "global-pull-secret-syncer-hbgj5" (UID: "503a2587-73cc-43a0-b058-242b6b7ec81b") : object "kube-system"/"original-pull-secret" not registered
Sep 06 10:30:05.818815 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:05.818514    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs podName:5ef2cd14-0fae-417c-8d8a-b407e0c03903 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:21.818506526 +0000 UTC m=+34.268287272 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs") pod "network-metrics-daemon-pgxvx" (UID: "5ef2cd14-0fae-417c-8d8a-b407e0c03903") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 10:30:05.919646 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:05.919613    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngkp5\" (UniqueName: \"kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5\") pod \"network-check-target-bcjj6\" (UID: \"96763caf-341a-40a7-ba78-c94b6dedebfd\") " pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:30:05.919809 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:05.919792    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 10:30:05.919859 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:05.919820    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 10:30:05.919859 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:05.919834    2557 projected.go:196] Error preparing data for projected volume kube-api-access-ngkp5 for pod openshift-network-diagnostics/network-check-target-bcjj6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 10:30:05.919944 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:05.919895    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5 podName:96763caf-341a-40a7-ba78-c94b6dedebfd nodeName:}" failed. No retries permitted until 2026-09-06 10:30:21.919878508 +0000 UTC m=+34.369659277 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ngkp5" (UniqueName: "kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5") pod "network-check-target-bcjj6" (UID: "96763caf-341a-40a7-ba78-c94b6dedebfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 10:30:06.030515 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:06.030445    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:30:06.030684 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:06.030614    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hbgj5" podUID="503a2587-73cc-43a0-b058-242b6b7ec81b"
Sep 06 10:30:06.031175 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:06.031021    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:30:06.031297 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:06.031190    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgxvx" podUID="5ef2cd14-0fae-417c-8d8a-b407e0c03903"
Sep 06 10:30:06.031297 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:06.031239    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:30:06.031405 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:06.031345    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcjj6" podUID="96763caf-341a-40a7-ba78-c94b6dedebfd"
Sep 06 10:30:08.031509 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:08.031461    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:30:08.031948 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:08.031578    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:30:08.031948 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:08.031585    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hbgj5" podUID="503a2587-73cc-43a0-b058-242b6b7ec81b"
Sep 06 10:30:08.031948 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:08.031609    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:30:08.031948 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:08.031697    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgxvx" podUID="5ef2cd14-0fae-417c-8d8a-b407e0c03903"
Sep 06 10:30:08.031948 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:08.031783    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcjj6" podUID="96763caf-341a-40a7-ba78-c94b6dedebfd"
Sep 06 10:30:09.192452 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:09.192187    2557 generic.go:356] "Generic (PLEG): container finished" podID="b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3" containerID="d744499fcb70206fdc2a8da3d2b2bd578a3a2027e83c62f07b51afbd9dc734a5" exitCode=0
Sep 06 10:30:09.193315 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:09.192256    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l57sp" event={"ID":"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3","Type":"ContainerDied","Data":"d744499fcb70206fdc2a8da3d2b2bd578a3a2027e83c62f07b51afbd9dc734a5"}
Sep 06 10:30:09.195320 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:09.195294    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tw474" event={"ID":"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb","Type":"ContainerStarted","Data":"9a53619809a766adbbd93827a884fd2f70e41d074bd9afe2f5e43f1ce337a5b6"}
Sep 06 10:30:09.195405 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:09.195373    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tw474" event={"ID":"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb","Type":"ContainerStarted","Data":"e0b0bc101caaee4f1ed538adfc16979f49a73cbd8af02c28a99d5e60089a2fa9"}
Sep 06 10:30:09.195461 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:09.195428    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tw474" event={"ID":"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb","Type":"ContainerStarted","Data":"970ef74ad33fcb8bf4c104b1a373cc590963df4c0d77415e6be0a4068716431c"}
Sep 06 10:30:09.197157 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:09.197133    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z72zd" event={"ID":"f27bdb46-dbcb-4639-ad28-3ed384e21ba6","Type":"ContainerStarted","Data":"b7edc81068af3090cdd14b9667f42f0b32b8897257b97e5df08d608e820084a3"}
Sep 06 10:30:09.198913 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:09.198892    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h" event={"ID":"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e","Type":"ContainerStarted","Data":"60d6d6846cf04b0a2d99ef8574361bfff3bad7f1dcd9c3e93eaa26274783b535"}
Sep 06 10:30:09.200537 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:09.200517    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kpqgt" event={"ID":"1fdbdca8-792e-4abe-8058-3cc6df71bedd","Type":"ContainerStarted","Data":"5653408522e75460fdd471ff1a84b98ef624dce160011bc42016f9a46cfd4f1d"}
Sep 06 10:30:09.201864 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:09.201845    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5kn44" event={"ID":"e037c2c9-f192-4742-b594-adecc8c04606","Type":"ContainerStarted","Data":"01296e839cb530688268e16063a247d48d17f8b758410e50623e9d77b5866e29"}
Sep 06 10:30:09.203341 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:09.203305    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t57dx" event={"ID":"8a42dde3-cdd1-49a0-86f1-c67b65ca9cee","Type":"ContainerStarted","Data":"26e1c156f9e5eacc33b649b8b3e53eeb239f48d75886d705cabd75f047c0f3a3"}
Sep 06 10:30:09.204870 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:09.204835    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-st44k" event={"ID":"d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3","Type":"ContainerStarted","Data":"777e9694c0db2d5f9c6753479e504246d6158dbea75258474ea64697ba86be4e"}
Sep 06 10:30:09.212410 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:09.212377    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-153.ec2.internal" podStartSLOduration=20.212367162 podStartE2EDuration="20.212367162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:29:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:29:53.172706117 +0000 UTC m=+5.622486886" watchObservedRunningTime="2026-09-06 10:30:09.212367162 +0000 UTC m=+21.662147929"
Sep 06 10:30:09.227283 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:09.227247    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-st44k" podStartSLOduration=3.622790303 podStartE2EDuration="21.227234599s" totalImagesPullingTime="17.604444296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:29:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:09.226838049 +0000 UTC m=+21.676618816" watchObservedRunningTime="2026-09-06 10:30:09.227234599 +0000 UTC m=+21.677015379"
Sep 06 10:30:09.241559 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:09.241526    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5kn44" podStartSLOduration=3.633968625 podStartE2EDuration="21.241514254s" totalImagesPullingTime="17.607545629s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:29:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:09.241477423 +0000 UTC m=+21.691258190" watchObservedRunningTime="2026-09-06 10:30:09.241514254 +0000 UTC m=+21.691295022"
Sep 06 10:30:09.280673 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:09.280639    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-kpqgt" podStartSLOduration=11.030816295 podStartE2EDuration="21.280626505s" totalImagesPullingTime="10.24981021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:29:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:09.260689638 +0000 UTC m=+21.710470417" watchObservedRunningTime="2026-09-06 10:30:09.280626505 +0000 UTC m=+21.730407273"
Sep 06 10:30:09.313481 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:09.313447    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-t57dx" podStartSLOduration=3.723238061 podStartE2EDuration="21.313435167s" totalImagesPullingTime="17.590197106s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:29:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:09.282544404 +0000 UTC m=+21.732325171" watchObservedRunningTime="2026-09-06 10:30:09.313435167 +0000 UTC m=+21.763215935"
Sep 06 10:30:09.313649 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:09.313631    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-z72zd" podStartSLOduration=3.670329008 podStartE2EDuration="21.313625942s" totalImagesPullingTime="17.643296934s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:29:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:09.313193459 +0000 UTC m=+21.762974227" watchObservedRunningTime="2026-09-06 10:30:09.313625942 +0000 UTC m=+21.763406710"
Sep 06 10:30:10.031329 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:10.031100    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:30:10.031528 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:10.031422    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgxvx" podUID="5ef2cd14-0fae-417c-8d8a-b407e0c03903"
Sep 06 10:30:10.031528 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:10.031100    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:30:10.031528 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:10.031219    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:30:10.031719 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:10.031548    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hbgj5" podUID="503a2587-73cc-43a0-b058-242b6b7ec81b"
Sep 06 10:30:10.031719 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:10.031623    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcjj6" podUID="96763caf-341a-40a7-ba78-c94b6dedebfd"
Sep 06 10:30:10.208401 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:10.208374    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-789d7" event={"ID":"8aef3159-3c8b-434c-91d6-c50e26c4c6ec","Type":"ContainerStarted","Data":"310458b49280cf71e6ca92bfe43c0c61d86a908e8d3d8bcbaa4fd367b3d64529"}
Sep 06 10:30:10.211873 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:10.211846    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tw474" event={"ID":"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb","Type":"ContainerStarted","Data":"84b13ead26be91571e049665dda456457b9233ba925a45686e58adecb2d67d29"}
Sep 06 10:30:10.211993 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:10.211928    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tw474" event={"ID":"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb","Type":"ContainerStarted","Data":"7703b383f3a87facfe328c936b1306195d40d0b44876e48e49ff6aed591982c6"}
Sep 06 10:30:10.211993 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:10.211980    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tw474" event={"ID":"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb","Type":"ContainerStarted","Data":"b55f0d0f84dbf42cbcaac1031ced430faf7fd24feaf0f8de8f5d5722ef807222"}
Sep 06 10:30:10.223005 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:10.222957    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-789d7" podStartSLOduration=4.630686195 podStartE2EDuration="22.222942018s" totalImagesPullingTime="17.592255823s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:29:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:10.222544929 +0000 UTC m=+22.672325698" watchObservedRunningTime="2026-09-06 10:30:10.222942018 +0000 UTC m=+22.672722785"
Sep 06 10:30:10.430311 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:10.430243    2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 06 10:30:11.031611 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:11.031450    2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-06T10:30:10.430263027Z","UUID":"4bf69e05-a080-4e96-a424-7eb835ef67af","Handler":null,"Name":"","Endpoint":""}
Sep 06 10:30:11.033298 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:11.033274    2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 06 10:30:11.033424 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:11.033311    2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 06 10:30:11.216305 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:11.216271    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h" event={"ID":"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e","Type":"ContainerStarted","Data":"12143e45350e411a396ab131b1a56411678e61f12fb37e75513d4708a24733d2"}
Sep 06 10:30:12.031006 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:12.030975    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:30:12.031160 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:12.031069    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcjj6" podUID="96763caf-341a-40a7-ba78-c94b6dedebfd"
Sep 06 10:30:12.031160 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:12.030953    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:30:12.031242 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:12.030953    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:30:12.031242 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:12.031175    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgxvx" podUID="5ef2cd14-0fae-417c-8d8a-b407e0c03903"
Sep 06 10:30:12.031242 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:12.031231    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hbgj5" podUID="503a2587-73cc-43a0-b058-242b6b7ec81b"
Sep 06 10:30:12.223007 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:12.222970    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tw474" event={"ID":"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb","Type":"ContainerStarted","Data":"176c08a2fb7557d96fd00d0a37e61ee7aaef0e8a2967df5db4134358aeb5005c"}
Sep 06 10:30:12.225339 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:12.225311    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h" event={"ID":"0950615a-b7ad-4c9c-af8b-5adcf5c3c44e","Type":"ContainerStarted","Data":"22a92eee51176a3dbb3704a523ca5e7cfa95544b664fbb8102b68aa4224a6052"}
Sep 06 10:30:12.242341 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:12.242302    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsv7h" podStartSLOduration=4.296296911 podStartE2EDuration="24.242290328s" totalImagesPullingTime="19.945993417s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:29:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:12.242120232 +0000 UTC m=+24.691901001" watchObservedRunningTime="2026-09-06 10:30:12.242290328 +0000 UTC m=+24.692071096"
Sep 06 10:30:13.107934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:13.107900    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-kpqgt"
Sep 06 10:30:13.108568 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:13.108549    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-kpqgt"
Sep 06 10:30:13.228431 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:13.228402    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-kpqgt"
Sep 06 10:30:13.228949 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:13.228906    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-kpqgt"
Sep 06 10:30:14.030667 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:14.030448    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:30:14.030802 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:14.030450    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:30:14.030802 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:14.030720    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hbgj5" podUID="503a2587-73cc-43a0-b058-242b6b7ec81b"
Sep 06 10:30:14.030802 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:14.030470    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:30:14.031124 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:14.031098    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcjj6" podUID="96763caf-341a-40a7-ba78-c94b6dedebfd"
Sep 06 10:30:14.031252 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:14.031215    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgxvx" podUID="5ef2cd14-0fae-417c-8d8a-b407e0c03903"
Sep 06 10:30:14.231465 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:14.231440    2557 generic.go:356] "Generic (PLEG): container finished" podID="b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3" containerID="37197ca122f5d0fa3937d2f9702d68e0e6821cc0772394539b7a04d77af40317" exitCode=0
Sep 06 10:30:14.232016 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:14.231521    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l57sp" event={"ID":"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3","Type":"ContainerDied","Data":"37197ca122f5d0fa3937d2f9702d68e0e6821cc0772394539b7a04d77af40317"}
Sep 06 10:30:14.235413 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:14.235391    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tw474" event={"ID":"cbcbc6f7-1879-4390-8b71-b2215b7fc9eb","Type":"ContainerStarted","Data":"9a3a336cb91481f25476ad2efa129faefa029a2ba2993d6210c2abb6cce626e2"}
Sep 06 10:30:14.280573 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:14.280536    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tw474" podStartSLOduration=8.645445595 podStartE2EDuration="26.280524228s" totalImagesPullingTime="17.635078633s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:29:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:14.280409623 +0000 UTC m=+26.730190391" watchObservedRunningTime="2026-09-06 10:30:14.280524228 +0000 UTC m=+26.730304995"
Sep 06 10:30:15.239615 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:15.239586    2557 generic.go:356] "Generic (PLEG): container finished" podID="b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3" containerID="4c0e329871bcbdc9b0f988b625b3d96825a33a7013c0cfd1590a013e46062809" exitCode=0
Sep 06 10:30:15.240210 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:15.239671    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l57sp" event={"ID":"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3","Type":"ContainerDied","Data":"4c0e329871bcbdc9b0f988b625b3d96825a33a7013c0cfd1590a013e46062809"}
Sep 06 10:30:15.240210 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:15.240171    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:30:15.240537 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:15.240379    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:30:15.240537 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:15.240457    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:30:15.254807 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:15.254789    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:30:15.255527 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:15.255510    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:30:15.636729 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:15.636540    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bcjj6"]
Sep 06 10:30:15.636880 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:15.636868    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:30:15.636972 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:15.636950    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcjj6" podUID="96763caf-341a-40a7-ba78-c94b6dedebfd"
Sep 06 10:30:15.639244 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:15.639220    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hbgj5"]
Sep 06 10:30:15.639366 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:15.639344    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:30:15.639465 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:15.639444    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hbgj5" podUID="503a2587-73cc-43a0-b058-242b6b7ec81b"
Sep 06 10:30:15.648476 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:15.648453    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pgxvx"]
Sep 06 10:30:15.648591 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:15.648573    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:30:15.648695 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:15.648676    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgxvx" podUID="5ef2cd14-0fae-417c-8d8a-b407e0c03903"
Sep 06 10:30:16.244149 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:16.244084    2557 generic.go:356] "Generic (PLEG): container finished" podID="b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3" containerID="98334df2051ab5e6c5d927c4863d7f6ba3f6d7a0e9b269e858067e293d4d174c" exitCode=0
Sep 06 10:30:16.244426 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:16.244166    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l57sp" event={"ID":"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3","Type":"ContainerDied","Data":"98334df2051ab5e6c5d927c4863d7f6ba3f6d7a0e9b269e858067e293d4d174c"}
Sep 06 10:30:17.030353 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:17.030324    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:30:17.030537 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:17.030437    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:30:17.030537 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:17.030446    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcjj6" podUID="96763caf-341a-40a7-ba78-c94b6dedebfd"
Sep 06 10:30:17.030663 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:17.030565    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hbgj5" podUID="503a2587-73cc-43a0-b058-242b6b7ec81b"
Sep 06 10:30:17.030663 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:17.030603    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:30:17.030778 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:17.030715    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgxvx" podUID="5ef2cd14-0fae-417c-8d8a-b407e0c03903"
Sep 06 10:30:19.030829 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:19.030790    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:30:19.031295 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:19.030794    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:30:19.031295 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:19.030919    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hbgj5" podUID="503a2587-73cc-43a0-b058-242b6b7ec81b"
Sep 06 10:30:19.031295 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:19.030805    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:30:19.031295 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:19.030997    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcjj6" podUID="96763caf-341a-40a7-ba78-c94b6dedebfd"
Sep 06 10:30:19.031295 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:19.031074    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgxvx" podUID="5ef2cd14-0fae-417c-8d8a-b407e0c03903"
Sep 06 10:30:21.030706 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:21.030674    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:30:21.031102 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:21.030781    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:30:21.031102 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:21.030808    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:30:21.031102 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:21.030797    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcjj6" podUID="96763caf-341a-40a7-ba78-c94b6dedebfd"
Sep 06 10:30:21.031102 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:21.030900    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hbgj5" podUID="503a2587-73cc-43a0-b058-242b6b7ec81b"
Sep 06 10:30:21.031102 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:21.030967    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgxvx" podUID="5ef2cd14-0fae-417c-8d8a-b407e0c03903"
Sep 06 10:30:21.260938 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:21.260736    2557 generic.go:356] "Generic (PLEG): container finished" podID="b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3" containerID="a78dace1630b0bf1bb77c010dba6ac7349ad9ed989a7d54ea35697ccb93be489" exitCode=0
Sep 06 10:30:21.260938 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:21.260828    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l57sp" event={"ID":"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3","Type":"ContainerDied","Data":"a78dace1630b0bf1bb77c010dba6ac7349ad9ed989a7d54ea35697ccb93be489"}
Sep 06 10:30:21.844880 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:21.844853    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret\") pod \"global-pull-secret-syncer-hbgj5\" (UID: \"503a2587-73cc-43a0-b058-242b6b7ec81b\") " pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:30:21.845066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:21.844890    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs\") pod \"network-metrics-daemon-pgxvx\" (UID: \"5ef2cd14-0fae-417c-8d8a-b407e0c03903\") " pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:30:21.845066 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:21.844987    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 10:30:21.845066 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:21.844997    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 10:30:21.845066 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:21.845033    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs podName:5ef2cd14-0fae-417c-8d8a-b407e0c03903 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:53.845020414 +0000 UTC m=+66.294801159 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs") pod "network-metrics-daemon-pgxvx" (UID: "5ef2cd14-0fae-417c-8d8a-b407e0c03903") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 10:30:21.845066 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:21.845056    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret podName:503a2587-73cc-43a0-b058-242b6b7ec81b nodeName:}" failed. No retries permitted until 2026-09-06 10:30:53.845038454 +0000 UTC m=+66.294819218 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret") pod "global-pull-secret-syncer-hbgj5" (UID: "503a2587-73cc-43a0-b058-242b6b7ec81b") : object "kube-system"/"original-pull-secret" not registered
Sep 06 10:30:21.945553 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:21.945532    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngkp5\" (UniqueName: \"kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5\") pod \"network-check-target-bcjj6\" (UID: \"96763caf-341a-40a7-ba78-c94b6dedebfd\") " pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:30:21.945694 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:21.945677    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 10:30:21.945739 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:21.945700    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 10:30:21.945739 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:21.945709    2557 projected.go:196] Error preparing data for projected volume kube-api-access-ngkp5 for pod openshift-network-diagnostics/network-check-target-bcjj6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 10:30:21.945802 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:21.945747    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5 podName:96763caf-341a-40a7-ba78-c94b6dedebfd nodeName:}" failed. No retries permitted until 2026-09-06 10:30:53.945736243 +0000 UTC m=+66.395517002 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ngkp5" (UniqueName: "kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5") pod "network-check-target-bcjj6" (UID: "96763caf-341a-40a7-ba78-c94b6dedebfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 10:30:22.266002 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.265975    2557 generic.go:356] "Generic (PLEG): container finished" podID="b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3" containerID="6126ef5fd8884169aaa99a7fde518f54ee49b785c664441d223f0d455d0c4d5b" exitCode=0
Sep 06 10:30:22.266428 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.266028    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l57sp" event={"ID":"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3","Type":"ContainerDied","Data":"6126ef5fd8884169aaa99a7fde518f54ee49b785c664441d223f0d455d0c4d5b"}
Sep 06 10:30:22.908503 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.908466    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-153.ec2.internal" event="NodeReady"
Sep 06 10:30:22.908642 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.908572    2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 06 10:30:22.940574 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.940552    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7jr4k"]
Sep 06 10:30:22.954873 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.954785    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-tr4gb"]
Sep 06 10:30:22.955013 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.954864    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7jr4k"
Sep 06 10:30:22.957852 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.957830    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 06 10:30:22.958100 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.958050    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 06 10:30:22.959041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.958990    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 06 10:30:22.959516 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.959308    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-cvqr7\""
Sep 06 10:30:22.959876 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.959810    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 06 10:30:22.970227 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.969473    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"]
Sep 06 10:30:22.970227 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.969768    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-tr4gb"
Sep 06 10:30:22.972568 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.972535    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 06 10:30:22.972771 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.972755    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 06 10:30:22.972838 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.972817    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-tk7w7\""
Sep 06 10:30:22.991304 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.991286    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"
Sep 06 10:30:22.991304 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.991297    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-p8s2s"]
Sep 06 10:30:22.993626 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.993609    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 06 10:30:22.994817 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.994756    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 06 10:30:22.994817 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.994769    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 06 10:30:22.995043 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.994869    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 06 10:30:22.995043 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:22.994999    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ld7ld\""
Sep 06 10:30:23.013684 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.013663    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b78f8f8f-f8nfj"]
Sep 06 10:30:23.013785 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.013772    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-p8s2s"
Sep 06 10:30:23.015990 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.015971    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 06 10:30:23.016085 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.015990    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 06 10:30:23.016085 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.016010    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-g7sqg\""
Sep 06 10:30:23.016085 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.016048    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 06 10:30:23.016257 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.016242    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 06 10:30:23.037642 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.037627    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm"]
Sep 06 10:30:23.037745 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.037731    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.040088 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.040076    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f2fgs\""
Sep 06 10:30:23.040179 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.040120    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 06 10:30:23.040179 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.040162    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 06 10:30:23.040415 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.040401    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 06 10:30:23.045841 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.045823    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 06 10:30:23.053979 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.053962    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/99fa6d2e-adc6-4bd8-b6b0-e73045c36405-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7jr4k\" (UID: \"99fa6d2e-adc6-4bd8-b6b0-e73045c36405\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7jr4k"
Sep 06 10:30:23.054044 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.054003    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjv92\" (UniqueName: \"kubernetes.io/projected/99fa6d2e-adc6-4bd8-b6b0-e73045c36405-kube-api-access-pjv92\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7jr4k\" (UID: \"99fa6d2e-adc6-4bd8-b6b0-e73045c36405\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7jr4k"
Sep 06 10:30:23.054044 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.054033    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/99fa6d2e-adc6-4bd8-b6b0-e73045c36405-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7jr4k\" (UID: \"99fa6d2e-adc6-4bd8-b6b0-e73045c36405\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7jr4k"
Sep 06 10:30:23.061782 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.061756    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-vnd8g"]
Sep 06 10:30:23.061861 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.061841    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:30:23.061902 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.061863    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:30:23.061902 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.061863    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm"
Sep 06 10:30:23.062093 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.062061    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:30:23.064193 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.064178    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 06 10:30:23.064279 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.064181    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h4kvp\""
Sep 06 10:30:23.064367 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.064308    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 06 10:30:23.064367 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.064313    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 06 10:30:23.064562 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.064546    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 06 10:30:23.064618 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.064568    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 06 10:30:23.064618 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.064554    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 06 10:30:23.064702 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.064680    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 06 10:30:23.064749 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.064710    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tdqz7\""
Sep 06 10:30:23.064821 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.064808    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bx9l2\""
Sep 06 10:30:23.081744 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.081727    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-9lt24"]
Sep 06 10:30:23.081867 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.081850    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.084240 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.084198    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 06 10:30:23.084333 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.084205    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 06 10:30:23.084333 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.084229    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 06 10:30:23.084509 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.084475    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 06 10:30:23.084576 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.084564    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-gvfjk\""
Sep 06 10:30:23.089643 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.089624    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 06 10:30:23.096228 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.096211    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-gmxdr"]
Sep 06 10:30:23.096319 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.096306    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:30:23.098677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.098660    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 06 10:30:23.098779 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.098762    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 06 10:30:23.098840 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.098801    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 06 10:30:23.098840 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.098834    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 06 10:30:23.099009 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.098991    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-tgrmb\""
Sep 06 10:30:23.103924 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.103904    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 06 10:30:23.112076 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.112062    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-78f4854998-kj5tp"]
Sep 06 10:30:23.112163 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.112151    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-gmxdr"
Sep 06 10:30:23.114475 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.114458    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-cjv8b\""
Sep 06 10:30:23.121791 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.121767    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kpqgt_1fdbdca8-792e-4abe-8058-3cc6df71bedd/konnectivity-agent/0.log" limit=1024
Sep 06 10:30:23.126114 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.126100    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-tr4gb"]
Sep 06 10:30:23.126176 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.126122    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7jr4k"]
Sep 06 10:30:23.126176 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.126135    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"]
Sep 06 10:30:23.126176 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.126149    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-p8s2s"]
Sep 06 10:30:23.126176 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.126167    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm"]
Sep 06 10:30:23.126299 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.126184    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-vnd8g"]
Sep 06 10:30:23.126299 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.126198    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dltkp"]
Sep 06 10:30:23.126299 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.126205    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:23.128693 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.128678    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 06 10:30:23.128823 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.128729    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 06 10:30:23.128823 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.128738    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 06 10:30:23.128823 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.128762    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 06 10:30:23.128952 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.128938    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 06 10:30:23.128992 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.128938    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-89vv4\""
Sep 06 10:30:23.129046 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.129029    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 06 10:30:23.138104 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.138089    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b78f8f8f-f8nfj"]
Sep 06 10:30:23.138184 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.138111    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-74t29"]
Sep 06 10:30:23.138224 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.138185    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:23.140439 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.140423    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 06 10:30:23.140548 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.140426    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 06 10:30:23.140548 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.140452    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-89vqh\""
Sep 06 10:30:23.150363 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.150346    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-74t29"
Sep 06 10:30:23.150469 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.150345    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-78f4854998-kj5tp"]
Sep 06 10:30:23.150569 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.150555    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-gmxdr"]
Sep 06 10:30:23.150611 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.150584    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-9lt24"]
Sep 06 10:30:23.150611 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.150600    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dltkp"]
Sep 06 10:30:23.150688 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.150616    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-74t29"]
Sep 06 10:30:23.154734 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.154692    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw4wf\" (UniqueName: \"kubernetes.io/projected/1462951b-ed57-4e82-8a1d-5d5f08f73ee9-kube-api-access-tw4wf\") pod \"service-ca-operator-859f94c4b-p8s2s\" (UID: \"1462951b-ed57-4e82-8a1d-5d5f08f73ee9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-p8s2s"
Sep 06 10:30:23.154857 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.154740    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kkzl\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-kube-api-access-6kkzl\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.154857 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.154767    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvtvt\" (UniqueName: \"kubernetes.io/projected/4e55abd8-3289-4fff-8d8f-34129da7fb69-kube-api-access-vvtvt\") pod \"cluster-monitoring-operator-6cdcc96965-zmk6j\" (UID: \"4e55abd8-3289-4fff-8d8f-34129da7fb69\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"
Sep 06 10:30:23.154857 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.154792    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69d7eb68-0175-46cd-9464-19e90cdcc1a3-image-registry-private-configuration\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.154857 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.154819    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69d7eb68-0175-46cd-9464-19e90cdcc1a3-installation-pull-secrets\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.154857 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.154844    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 06 10:30:23.155117 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.154851    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/99fa6d2e-adc6-4bd8-b6b0-e73045c36405-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7jr4k\" (UID: \"99fa6d2e-adc6-4bd8-b6b0-e73045c36405\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7jr4k"
Sep 06 10:30:23.155117 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.154940    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgncd\" (UniqueName: \"kubernetes.io/projected/9dc8c2c3-d63b-4ae4-8458-e37779164ff2-kube-api-access-cgncd\") pod \"volume-data-source-validator-695c895bd6-tr4gb\" (UID: \"9dc8c2c3-d63b-4ae4-8458-e37779164ff2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-tr4gb"
Sep 06 10:30:23.155117 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.154974    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-certificates\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.155117 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.154999    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69d7eb68-0175-46cd-9464-19e90cdcc1a3-trusted-ca\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.155117 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.155005    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-66txg\""
Sep 06 10:30:23.155117 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.155030    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-zmk6j\" (UID: \"4e55abd8-3289-4fff-8d8f-34129da7fb69\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"
Sep 06 10:30:23.155117 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.155062    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/99fa6d2e-adc6-4bd8-b6b0-e73045c36405-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7jr4k\" (UID: \"99fa6d2e-adc6-4bd8-b6b0-e73045c36405\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7jr4k"
Sep 06 10:30:23.155117 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.155087    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1462951b-ed57-4e82-8a1d-5d5f08f73ee9-config\") pod \"service-ca-operator-859f94c4b-p8s2s\" (UID: \"1462951b-ed57-4e82-8a1d-5d5f08f73ee9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-p8s2s"
Sep 06 10:30:23.155117 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.155091    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 06 10:30:23.155117 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.155109    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-bound-sa-token\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.155443 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.155160    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjv92\" (UniqueName: \"kubernetes.io/projected/99fa6d2e-adc6-4bd8-b6b0-e73045c36405-kube-api-access-pjv92\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7jr4k\" (UID: \"99fa6d2e-adc6-4bd8-b6b0-e73045c36405\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7jr4k"
Sep 06 10:30:23.155443 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.155183    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69d7eb68-0175-46cd-9464-19e90cdcc1a3-ca-trust-extracted\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.155443 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.155264    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 06 10:30:23.155443 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.155406    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4e55abd8-3289-4fff-8d8f-34129da7fb69-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-zmk6j\" (UID: \"4e55abd8-3289-4fff-8d8f-34129da7fb69\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"
Sep 06 10:30:23.155637 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.155443    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1462951b-ed57-4e82-8a1d-5d5f08f73ee9-serving-cert\") pod \"service-ca-operator-859f94c4b-p8s2s\" (UID: \"1462951b-ed57-4e82-8a1d-5d5f08f73ee9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-p8s2s"
Sep 06 10:30:23.155637 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.155471    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.155726 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.155662    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/99fa6d2e-adc6-4bd8-b6b0-e73045c36405-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7jr4k\" (UID: \"99fa6d2e-adc6-4bd8-b6b0-e73045c36405\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7jr4k"
Sep 06 10:30:23.159855 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.159803    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/99fa6d2e-adc6-4bd8-b6b0-e73045c36405-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7jr4k\" (UID: \"99fa6d2e-adc6-4bd8-b6b0-e73045c36405\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7jr4k"
Sep 06 10:30:23.163902 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.163879    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjv92\" (UniqueName: \"kubernetes.io/projected/99fa6d2e-adc6-4bd8-b6b0-e73045c36405-kube-api-access-pjv92\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7jr4k\" (UID: \"99fa6d2e-adc6-4bd8-b6b0-e73045c36405\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7jr4k"
Sep 06 10:30:23.209049 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.209028    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kpqgt_1fdbdca8-792e-4abe-8058-3cc6df71bedd/konnectivity-agent/0.log" limit=1024
Sep 06 10:30:23.256482 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.256458    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-stats-auth\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:23.256589 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.256483    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ec03af21-0135-4ba1-b4be-06adf19748ed-config-volume\") pod \"dns-default-dltkp\" (UID: \"ec03af21-0135-4ba1-b4be-06adf19748ed\") " pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:23.256589 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.256516    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc7m7\" (UniqueName: \"kubernetes.io/projected/0f62f367-e8fd-4f68-95ba-50376d52e32a-kube-api-access-nc7m7\") pod \"ingress-canary-74t29\" (UID: \"0f62f367-e8fd-4f68-95ba-50376d52e32a\") " pod="openshift-ingress-canary/ingress-canary-74t29"
Sep 06 10:30:23.256589 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.256537    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-certificates\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.256589 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.256552    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69d7eb68-0175-46cd-9464-19e90cdcc1a3-trusted-ca\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.256788 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.256597    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/679d9837-653d-4ac4-b786-dd55ac8a5e06-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-vnd8g\" (UID: \"679d9837-653d-4ac4-b786-dd55ac8a5e06\") " pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.256788 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.256630    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/938e293f-7414-4bdd-946f-0b463ddc8ecb-serving-cert\") pod \"console-operator-85b454874b-9lt24\" (UID: \"938e293f-7414-4bdd-946f-0b463ddc8ecb\") " pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:30:23.256788 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.256647    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/679d9837-653d-4ac4-b786-dd55ac8a5e06-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-vnd8g\" (UID: \"679d9837-653d-4ac4-b786-dd55ac8a5e06\") " pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.256788 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.256662    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f62f367-e8fd-4f68-95ba-50376d52e32a-cert\") pod \"ingress-canary-74t29\" (UID: \"0f62f367-e8fd-4f68-95ba-50376d52e32a\") " pod="openshift-ingress-canary/ingress-canary-74t29"
Sep 06 10:30:23.256788 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.256718    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls\") pod \"dns-default-dltkp\" (UID: \"ec03af21-0135-4ba1-b4be-06adf19748ed\") " pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:23.256788 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.256750    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbtf4\" (UniqueName: \"kubernetes.io/projected/679d9837-653d-4ac4-b786-dd55ac8a5e06-kube-api-access-nbtf4\") pod \"insights-operator-76bcd6ccf5-vnd8g\" (UID: \"679d9837-653d-4ac4-b786-dd55ac8a5e06\") " pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.256788 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.256777    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1462951b-ed57-4e82-8a1d-5d5f08f73ee9-serving-cert\") pod \"service-ca-operator-859f94c4b-p8s2s\" (UID: \"1462951b-ed57-4e82-8a1d-5d5f08f73ee9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-p8s2s"
Sep 06 10:30:23.257112 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.256801    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.257112 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.256825    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:23.257112 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.256847    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj6b4\" (UniqueName: \"kubernetes.io/projected/ec03af21-0135-4ba1-b4be-06adf19748ed-kube-api-access-gj6b4\") pod \"dns-default-dltkp\" (UID: \"ec03af21-0135-4ba1-b4be-06adf19748ed\") " pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:23.257112 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.256889    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 06 10:30:23.257112 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.256903    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b78f8f8f-f8nfj: secret "image-registry-tls" not found
Sep 06 10:30:23.257112 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.256906    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw4wf\" (UniqueName: \"kubernetes.io/projected/1462951b-ed57-4e82-8a1d-5d5f08f73ee9-kube-api-access-tw4wf\") pod \"service-ca-operator-859f94c4b-p8s2s\" (UID: \"1462951b-ed57-4e82-8a1d-5d5f08f73ee9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-p8s2s"
Sep 06 10:30:23.257112 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.256938    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kkzl\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-kube-api-access-6kkzl\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.257112 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.256958    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls podName:69d7eb68-0175-46cd-9464-19e90cdcc1a3 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:23.7569424 +0000 UTC m=+36.206723153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls") pod "image-registry-7b78f8f8f-f8nfj" (UID: "69d7eb68-0175-46cd-9464-19e90cdcc1a3") : secret "image-registry-tls" not found
Sep 06 10:30:23.257112 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.256984    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/938e293f-7414-4bdd-946f-0b463ddc8ecb-trusted-ca\") pod \"console-operator-85b454874b-9lt24\" (UID: \"938e293f-7414-4bdd-946f-0b463ddc8ecb\") " pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:30:23.257112 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257011    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69d7eb68-0175-46cd-9464-19e90cdcc1a3-image-registry-private-configuration\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.257112 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257036    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-default-certificate\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:23.257112 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257060    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbrlm\" (UniqueName: \"kubernetes.io/projected/938e293f-7414-4bdd-946f-0b463ddc8ecb-kube-api-access-xbrlm\") pod \"console-operator-85b454874b-9lt24\" (UID: \"938e293f-7414-4bdd-946f-0b463ddc8ecb\") " pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:30:23.257112 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257097    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll268\" (UniqueName: \"kubernetes.io/projected/a44ce5d7-ab56-4b70-9f5f-8b312a5bb8ac-kube-api-access-ll268\") pod \"network-check-source-685456655f-gmxdr\" (UID: \"a44ce5d7-ab56-4b70-9f5f-8b312a5bb8ac\") " pod="openshift-network-diagnostics/network-check-source-685456655f-gmxdr"
Sep 06 10:30:23.257112 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257098    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-certificates\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.257778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257135    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:23.257778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257253    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/938e293f-7414-4bdd-946f-0b463ddc8ecb-config\") pod \"console-operator-85b454874b-9lt24\" (UID: \"938e293f-7414-4bdd-946f-0b463ddc8ecb\") " pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:30:23.257778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257291    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-zmk6j\" (UID: \"4e55abd8-3289-4fff-8d8f-34129da7fb69\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"
Sep 06 10:30:23.257778 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.257365    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 06 10:30:23.257778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257365    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/679d9837-653d-4ac4-b786-dd55ac8a5e06-serving-cert\") pod \"insights-operator-76bcd6ccf5-vnd8g\" (UID: \"679d9837-653d-4ac4-b786-dd55ac8a5e06\") " pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.257778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257406    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1462951b-ed57-4e82-8a1d-5d5f08f73ee9-config\") pod \"service-ca-operator-859f94c4b-p8s2s\" (UID: \"1462951b-ed57-4e82-8a1d-5d5f08f73ee9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-p8s2s"
Sep 06 10:30:23.257778 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.257419    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls podName:4e55abd8-3289-4fff-8d8f-34129da7fb69 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:23.757403619 +0000 UTC m=+36.207184592 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-zmk6j" (UID: "4e55abd8-3289-4fff-8d8f-34129da7fb69") : secret "cluster-monitoring-operator-tls" not found
Sep 06 10:30:23.257778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257432    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69d7eb68-0175-46cd-9464-19e90cdcc1a3-trusted-ca\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.257778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257444    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-bound-sa-token\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.257778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257475    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69d7eb68-0175-46cd-9464-19e90cdcc1a3-ca-trust-extracted\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.257778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257510    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4e55abd8-3289-4fff-8d8f-34129da7fb69-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-zmk6j\" (UID: \"4e55abd8-3289-4fff-8d8f-34129da7fb69\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"
Sep 06 10:30:23.257778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257537    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c84rd\" (UniqueName: \"kubernetes.io/projected/c2cbcefd-af14-4314-a52c-674e6775c298-kube-api-access-c84rd\") pod \"cluster-samples-operator-7cdcfcf76c-7jsrm\" (UID: \"c2cbcefd-af14-4314-a52c-674e6775c298\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm"
Sep 06 10:30:23.257778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257575    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-7jsrm\" (UID: \"c2cbcefd-af14-4314-a52c-674e6775c298\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm"
Sep 06 10:30:23.257778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257609    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvtvt\" (UniqueName: \"kubernetes.io/projected/4e55abd8-3289-4fff-8d8f-34129da7fb69-kube-api-access-vvtvt\") pod \"cluster-monitoring-operator-6cdcc96965-zmk6j\" (UID: \"4e55abd8-3289-4fff-8d8f-34129da7fb69\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"
Sep 06 10:30:23.257778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257648    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69d7eb68-0175-46cd-9464-19e90cdcc1a3-installation-pull-secrets\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.258518 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257679    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgncd\" (UniqueName: \"kubernetes.io/projected/9dc8c2c3-d63b-4ae4-8458-e37779164ff2-kube-api-access-cgncd\") pod \"volume-data-source-validator-695c895bd6-tr4gb\" (UID: \"9dc8c2c3-d63b-4ae4-8458-e37779164ff2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-tr4gb"
Sep 06 10:30:23.258518 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257705    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/679d9837-653d-4ac4-b786-dd55ac8a5e06-tmp\") pod \"insights-operator-76bcd6ccf5-vnd8g\" (UID: \"679d9837-653d-4ac4-b786-dd55ac8a5e06\") " pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.258518 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257729    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/679d9837-653d-4ac4-b786-dd55ac8a5e06-snapshots\") pod \"insights-operator-76bcd6ccf5-vnd8g\" (UID: \"679d9837-653d-4ac4-b786-dd55ac8a5e06\") " pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.258518 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257731    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69d7eb68-0175-46cd-9464-19e90cdcc1a3-ca-trust-extracted\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.258518 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257768    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j7cz\" (UniqueName: \"kubernetes.io/projected/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-kube-api-access-2j7cz\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:23.258518 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257801    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec03af21-0135-4ba1-b4be-06adf19748ed-tmp-dir\") pod \"dns-default-dltkp\" (UID: \"ec03af21-0135-4ba1-b4be-06adf19748ed\") " pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:23.258518 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.257941    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1462951b-ed57-4e82-8a1d-5d5f08f73ee9-config\") pod \"service-ca-operator-859f94c4b-p8s2s\" (UID: \"1462951b-ed57-4e82-8a1d-5d5f08f73ee9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-p8s2s"
Sep 06 10:30:23.258518 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.258268    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4e55abd8-3289-4fff-8d8f-34129da7fb69-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-zmk6j\" (UID: \"4e55abd8-3289-4fff-8d8f-34129da7fb69\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"
Sep 06 10:30:23.259056 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.259035    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1462951b-ed57-4e82-8a1d-5d5f08f73ee9-serving-cert\") pod \"service-ca-operator-859f94c4b-p8s2s\" (UID: \"1462951b-ed57-4e82-8a1d-5d5f08f73ee9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-p8s2s"
Sep 06 10:30:23.259900 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.259883    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69d7eb68-0175-46cd-9464-19e90cdcc1a3-image-registry-private-configuration\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.260062 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.260045    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69d7eb68-0175-46cd-9464-19e90cdcc1a3-installation-pull-secrets\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.265922 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.265900    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kkzl\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-kube-api-access-6kkzl\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.266113 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.266094    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-bound-sa-token\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.266604 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.266125    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw4wf\" (UniqueName: \"kubernetes.io/projected/1462951b-ed57-4e82-8a1d-5d5f08f73ee9-kube-api-access-tw4wf\") pod \"service-ca-operator-859f94c4b-p8s2s\" (UID: \"1462951b-ed57-4e82-8a1d-5d5f08f73ee9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-p8s2s"
Sep 06 10:30:23.266731 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.266711    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvtvt\" (UniqueName: \"kubernetes.io/projected/4e55abd8-3289-4fff-8d8f-34129da7fb69-kube-api-access-vvtvt\") pod \"cluster-monitoring-operator-6cdcc96965-zmk6j\" (UID: \"4e55abd8-3289-4fff-8d8f-34129da7fb69\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"
Sep 06 10:30:23.268705 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.268688    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgncd\" (UniqueName: \"kubernetes.io/projected/9dc8c2c3-d63b-4ae4-8458-e37779164ff2-kube-api-access-cgncd\") pod \"volume-data-source-validator-695c895bd6-tr4gb\" (UID: \"9dc8c2c3-d63b-4ae4-8458-e37779164ff2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-tr4gb"
Sep 06 10:30:23.271454 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.271433    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l57sp" event={"ID":"b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3","Type":"ContainerStarted","Data":"34030da9e3c16634aa7fdad72fa69a52d6c923708914dfe8ac51d02c68779e35"}
Sep 06 10:30:23.271915 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.271902    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7jr4k"
Sep 06 10:30:23.279844 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.279826    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-tr4gb"
Sep 06 10:30:23.291706 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.291665    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-l57sp" podStartSLOduration=7.05123776 podStartE2EDuration="35.291648846s" totalImagesPullingTime="28.240411086s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:29:48 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:23.29050644 +0000 UTC m=+35.740287201" watchObservedRunningTime="2026-09-06 10:30:23.291648846 +0000 UTC m=+35.741429614"
Sep 06 10:30:23.321601 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.321569    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-p8s2s"
Sep 06 10:30:23.359934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.358680    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll268\" (UniqueName: \"kubernetes.io/projected/a44ce5d7-ab56-4b70-9f5f-8b312a5bb8ac-kube-api-access-ll268\") pod \"network-check-source-685456655f-gmxdr\" (UID: \"a44ce5d7-ab56-4b70-9f5f-8b312a5bb8ac\") " pod="openshift-network-diagnostics/network-check-source-685456655f-gmxdr"
Sep 06 10:30:23.359934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.358846    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:23.359934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.358886    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/938e293f-7414-4bdd-946f-0b463ddc8ecb-config\") pod \"console-operator-85b454874b-9lt24\" (UID: \"938e293f-7414-4bdd-946f-0b463ddc8ecb\") " pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:30:23.359934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.358932    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/679d9837-653d-4ac4-b786-dd55ac8a5e06-serving-cert\") pod \"insights-operator-76bcd6ccf5-vnd8g\" (UID: \"679d9837-653d-4ac4-b786-dd55ac8a5e06\") " pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.359934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.358973    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c84rd\" (UniqueName: \"kubernetes.io/projected/c2cbcefd-af14-4314-a52c-674e6775c298-kube-api-access-c84rd\") pod \"cluster-samples-operator-7cdcfcf76c-7jsrm\" (UID: \"c2cbcefd-af14-4314-a52c-674e6775c298\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm"
Sep 06 10:30:23.359934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.359010    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-7jsrm\" (UID: \"c2cbcefd-af14-4314-a52c-674e6775c298\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm"
Sep 06 10:30:23.359934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.359038    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/679d9837-653d-4ac4-b786-dd55ac8a5e06-tmp\") pod \"insights-operator-76bcd6ccf5-vnd8g\" (UID: \"679d9837-653d-4ac4-b786-dd55ac8a5e06\") " pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.359934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.359062    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/679d9837-653d-4ac4-b786-dd55ac8a5e06-snapshots\") pod \"insights-operator-76bcd6ccf5-vnd8g\" (UID: \"679d9837-653d-4ac4-b786-dd55ac8a5e06\") " pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.359934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.359087    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j7cz\" (UniqueName: \"kubernetes.io/projected/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-kube-api-access-2j7cz\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:23.359934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.359116    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec03af21-0135-4ba1-b4be-06adf19748ed-tmp-dir\") pod \"dns-default-dltkp\" (UID: \"ec03af21-0135-4ba1-b4be-06adf19748ed\") " pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:23.359934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.359145    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-stats-auth\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:23.359934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.359169    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ec03af21-0135-4ba1-b4be-06adf19748ed-config-volume\") pod \"dns-default-dltkp\" (UID: \"ec03af21-0135-4ba1-b4be-06adf19748ed\") " pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:23.359934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.359194    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc7m7\" (UniqueName: \"kubernetes.io/projected/0f62f367-e8fd-4f68-95ba-50376d52e32a-kube-api-access-nc7m7\") pod \"ingress-canary-74t29\" (UID: \"0f62f367-e8fd-4f68-95ba-50376d52e32a\") " pod="openshift-ingress-canary/ingress-canary-74t29"
Sep 06 10:30:23.359934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.359223    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/679d9837-653d-4ac4-b786-dd55ac8a5e06-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-vnd8g\" (UID: \"679d9837-653d-4ac4-b786-dd55ac8a5e06\") " pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.359934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.359256    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/938e293f-7414-4bdd-946f-0b463ddc8ecb-serving-cert\") pod \"console-operator-85b454874b-9lt24\" (UID: \"938e293f-7414-4bdd-946f-0b463ddc8ecb\") " pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:30:23.359934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.359280    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/679d9837-653d-4ac4-b786-dd55ac8a5e06-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-vnd8g\" (UID: \"679d9837-653d-4ac4-b786-dd55ac8a5e06\") " pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.362981 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.359305    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f62f367-e8fd-4f68-95ba-50376d52e32a-cert\") pod \"ingress-canary-74t29\" (UID: \"0f62f367-e8fd-4f68-95ba-50376d52e32a\") " pod="openshift-ingress-canary/ingress-canary-74t29"
Sep 06 10:30:23.362981 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.359349    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls\") pod \"dns-default-dltkp\" (UID: \"ec03af21-0135-4ba1-b4be-06adf19748ed\") " pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:23.362981 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.359374    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbtf4\" (UniqueName: \"kubernetes.io/projected/679d9837-653d-4ac4-b786-dd55ac8a5e06-kube-api-access-nbtf4\") pod \"insights-operator-76bcd6ccf5-vnd8g\" (UID: \"679d9837-653d-4ac4-b786-dd55ac8a5e06\") " pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.362981 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.359412    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:23.362981 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.359438    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj6b4\" (UniqueName: \"kubernetes.io/projected/ec03af21-0135-4ba1-b4be-06adf19748ed-kube-api-access-gj6b4\") pod \"dns-default-dltkp\" (UID: \"ec03af21-0135-4ba1-b4be-06adf19748ed\") " pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:23.362981 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.360903    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 06 10:30:23.362981 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.360992    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs podName:0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c nodeName:}" failed. No retries permitted until 2026-09-06 10:30:23.860970941 +0000 UTC m=+36.310751714 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs") pod "router-default-78f4854998-kj5tp" (UID: "0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c") : secret "router-metrics-certs-default" not found
Sep 06 10:30:23.362981 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.361090    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/679d9837-653d-4ac4-b786-dd55ac8a5e06-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-vnd8g\" (UID: \"679d9837-653d-4ac4-b786-dd55ac8a5e06\") " pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.362981 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.361296    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec03af21-0135-4ba1-b4be-06adf19748ed-tmp-dir\") pod \"dns-default-dltkp\" (UID: \"ec03af21-0135-4ba1-b4be-06adf19748ed\") " pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:23.362981 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.361509    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/679d9837-653d-4ac4-b786-dd55ac8a5e06-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-vnd8g\" (UID: \"679d9837-653d-4ac4-b786-dd55ac8a5e06\") " pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.362981 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.361682    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle podName:0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c nodeName:}" failed. No retries permitted until 2026-09-06 10:30:23.861659711 +0000 UTC m=+36.311440481 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle") pod "router-default-78f4854998-kj5tp" (UID: "0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c") : configmap references non-existent config key: service-ca.crt
Sep 06 10:30:23.362981 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.362293    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/938e293f-7414-4bdd-946f-0b463ddc8ecb-trusted-ca\") pod \"console-operator-85b454874b-9lt24\" (UID: \"938e293f-7414-4bdd-946f-0b463ddc8ecb\") " pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:30:23.362981 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.362551    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ec03af21-0135-4ba1-b4be-06adf19748ed-config-volume\") pod \"dns-default-dltkp\" (UID: \"ec03af21-0135-4ba1-b4be-06adf19748ed\") " pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:23.362981 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.362711    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 06 10:30:23.362981 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.362762    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls podName:c2cbcefd-af14-4314-a52c-674e6775c298 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:23.862745205 +0000 UTC m=+36.312525968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-7jsrm" (UID: "c2cbcefd-af14-4314-a52c-674e6775c298") : secret "samples-operator-tls" not found
Sep 06 10:30:23.362981 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.362838    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/938e293f-7414-4bdd-946f-0b463ddc8ecb-config\") pod \"console-operator-85b454874b-9lt24\" (UID: \"938e293f-7414-4bdd-946f-0b463ddc8ecb\") " pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:30:23.363798 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.362910    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 10:30:23.363798 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.362956    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls podName:ec03af21-0135-4ba1-b4be-06adf19748ed nodeName:}" failed. No retries permitted until 2026-09-06 10:30:23.862941788 +0000 UTC m=+36.312722534 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls") pod "dns-default-dltkp" (UID: "ec03af21-0135-4ba1-b4be-06adf19748ed") : secret "dns-default-metrics-tls" not found
Sep 06 10:30:23.363798 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.363025    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/679d9837-653d-4ac4-b786-dd55ac8a5e06-tmp\") pod \"insights-operator-76bcd6ccf5-vnd8g\" (UID: \"679d9837-653d-4ac4-b786-dd55ac8a5e06\") " pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.364833 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.364360    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f62f367-e8fd-4f68-95ba-50376d52e32a-cert\") pod \"ingress-canary-74t29\" (UID: \"0f62f367-e8fd-4f68-95ba-50376d52e32a\") " pod="openshift-ingress-canary/ingress-canary-74t29"
Sep 06 10:30:23.364833 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.359468    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/938e293f-7414-4bdd-946f-0b463ddc8ecb-trusted-ca\") pod \"console-operator-85b454874b-9lt24\" (UID: \"938e293f-7414-4bdd-946f-0b463ddc8ecb\") " pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:30:23.364833 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.364509    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-default-certificate\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:23.364833 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.364536    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbrlm\" (UniqueName: \"kubernetes.io/projected/938e293f-7414-4bdd-946f-0b463ddc8ecb-kube-api-access-xbrlm\") pod \"console-operator-85b454874b-9lt24\" (UID: \"938e293f-7414-4bdd-946f-0b463ddc8ecb\") " pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:30:23.364833 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.364650    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/679d9837-653d-4ac4-b786-dd55ac8a5e06-serving-cert\") pod \"insights-operator-76bcd6ccf5-vnd8g\" (UID: \"679d9837-653d-4ac4-b786-dd55ac8a5e06\") " pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.364833 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.364777    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-stats-auth\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:23.365996 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.365957    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/679d9837-653d-4ac4-b786-dd55ac8a5e06-snapshots\") pod \"insights-operator-76bcd6ccf5-vnd8g\" (UID: \"679d9837-653d-4ac4-b786-dd55ac8a5e06\") " pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.366624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.366588    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/938e293f-7414-4bdd-946f-0b463ddc8ecb-serving-cert\") pod \"console-operator-85b454874b-9lt24\" (UID: \"938e293f-7414-4bdd-946f-0b463ddc8ecb\") " pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:30:23.367701 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.367404    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-default-certificate\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:23.370098 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.370049    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbtf4\" (UniqueName: \"kubernetes.io/projected/679d9837-653d-4ac4-b786-dd55ac8a5e06-kube-api-access-nbtf4\") pod \"insights-operator-76bcd6ccf5-vnd8g\" (UID: \"679d9837-653d-4ac4-b786-dd55ac8a5e06\") " pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.371746 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.371714    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c84rd\" (UniqueName: \"kubernetes.io/projected/c2cbcefd-af14-4314-a52c-674e6775c298-kube-api-access-c84rd\") pod \"cluster-samples-operator-7cdcfcf76c-7jsrm\" (UID: \"c2cbcefd-af14-4314-a52c-674e6775c298\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm"
Sep 06 10:30:23.372091 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.372048    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll268\" (UniqueName: \"kubernetes.io/projected/a44ce5d7-ab56-4b70-9f5f-8b312a5bb8ac-kube-api-access-ll268\") pod \"network-check-source-685456655f-gmxdr\" (UID: \"a44ce5d7-ab56-4b70-9f5f-8b312a5bb8ac\") " pod="openshift-network-diagnostics/network-check-source-685456655f-gmxdr"
Sep 06 10:30:23.377452 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.377425    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j7cz\" (UniqueName: \"kubernetes.io/projected/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-kube-api-access-2j7cz\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:23.377603 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.377583    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc7m7\" (UniqueName: \"kubernetes.io/projected/0f62f367-e8fd-4f68-95ba-50376d52e32a-kube-api-access-nc7m7\") pod \"ingress-canary-74t29\" (UID: \"0f62f367-e8fd-4f68-95ba-50376d52e32a\") " pod="openshift-ingress-canary/ingress-canary-74t29"
Sep 06 10:30:23.377667 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.377610    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbrlm\" (UniqueName: \"kubernetes.io/projected/938e293f-7414-4bdd-946f-0b463ddc8ecb-kube-api-access-xbrlm\") pod \"console-operator-85b454874b-9lt24\" (UID: \"938e293f-7414-4bdd-946f-0b463ddc8ecb\") " pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:30:23.378084 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.378062    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj6b4\" (UniqueName: \"kubernetes.io/projected/ec03af21-0135-4ba1-b4be-06adf19748ed-kube-api-access-gj6b4\") pod \"dns-default-dltkp\" (UID: \"ec03af21-0135-4ba1-b4be-06adf19748ed\") " pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:23.394144 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.394114    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g"
Sep 06 10:30:23.405115 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.405093    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:30:23.419379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.419248    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-gmxdr"
Sep 06 10:30:23.419601 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.419571    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kpqgt_1fdbdca8-792e-4abe-8058-3cc6df71bedd/konnectivity-agent/0.log" limit=1024
Sep 06 10:30:23.466605 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.466542    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-74t29"
Sep 06 10:30:23.471114 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.471046    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kpqgt_1fdbdca8-792e-4abe-8058-3cc6df71bedd/konnectivity-agent/0.log" limit=1024
Sep 06 10:30:23.567437 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.567411    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-tr4gb"]
Sep 06 10:30:23.568616 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.568466    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7jr4k"]
Sep 06 10:30:23.570364 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:23.570338    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99fa6d2e_adc6_4bd8_b6b0_e73045c36405.slice/crio-389a6ca94336ee2568ca183e8fa26ded8c73274b80e7f5ff9a6a8657879e0d43 WatchSource:0}: Error finding container 389a6ca94336ee2568ca183e8fa26ded8c73274b80e7f5ff9a6a8657879e0d43: Status 404 returned error can't find the container with id 389a6ca94336ee2568ca183e8fa26ded8c73274b80e7f5ff9a6a8657879e0d43
Sep 06 10:30:23.570977 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:23.570956    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dc8c2c3_d63b_4ae4_8458_e37779164ff2.slice/crio-afe404942f12bd071dae3b71c2983cfae25df5c449cde2b5f2c039a1b58b4fa2 WatchSource:0}: Error finding container afe404942f12bd071dae3b71c2983cfae25df5c449cde2b5f2c039a1b58b4fa2: Status 404 returned error can't find the container with id afe404942f12bd071dae3b71c2983cfae25df5c449cde2b5f2c039a1b58b4fa2
Sep 06 10:30:23.620841 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.620819    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-p8s2s"]
Sep 06 10:30:23.623689 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:23.623669    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1462951b_ed57_4e82_8a1d_5d5f08f73ee9.slice/crio-4c2044a1296950e03708d900e716ece79ccfa8f89fdcacc1db7cb41f1a41bbdb WatchSource:0}: Error finding container 4c2044a1296950e03708d900e716ece79ccfa8f89fdcacc1db7cb41f1a41bbdb: Status 404 returned error can't find the container with id 4c2044a1296950e03708d900e716ece79ccfa8f89fdcacc1db7cb41f1a41bbdb
Sep 06 10:30:23.675612 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.675596    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-vnd8g"]
Sep 06 10:30:23.685812 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.685791    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-9lt24"]
Sep 06 10:30:23.687963 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:23.687943    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod938e293f_7414_4bdd_946f_0b463ddc8ecb.slice/crio-59ec49ff61a6767047735f07bb782de7b7b640fb3e04a64670bd1fbfb1fc6b5a WatchSource:0}: Error finding container 59ec49ff61a6767047735f07bb782de7b7b640fb3e04a64670bd1fbfb1fc6b5a: Status 404 returned error can't find the container with id 59ec49ff61a6767047735f07bb782de7b7b640fb3e04a64670bd1fbfb1fc6b5a
Sep 06 10:30:23.689554 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:23.689534    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod679d9837_653d_4ac4_b786_dd55ac8a5e06.slice/crio-0686f2b8252de9f395efd0d36e609db7ba670c73c4d8b35d72d2ae067157e1d6 WatchSource:0}: Error finding container 0686f2b8252de9f395efd0d36e609db7ba670c73c4d8b35d72d2ae067157e1d6: Status 404 returned error can't find the container with id 0686f2b8252de9f395efd0d36e609db7ba670c73c4d8b35d72d2ae067157e1d6
Sep 06 10:30:23.705601 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.705578    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-gmxdr"]
Sep 06 10:30:23.708275 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:23.708255    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda44ce5d7_ab56_4b70_9f5f_8b312a5bb8ac.slice/crio-11896d8833fa0a8cc0b0c0d120bf00ea294f9e855b4c0b158bb328b60b205533 WatchSource:0}: Error finding container 11896d8833fa0a8cc0b0c0d120bf00ea294f9e855b4c0b158bb328b60b205533: Status 404 returned error can't find the container with id 11896d8833fa0a8cc0b0c0d120bf00ea294f9e855b4c0b158bb328b60b205533
Sep 06 10:30:23.745874 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.745854    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-74t29"]
Sep 06 10:30:23.748230 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:23.748209    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f62f367_e8fd_4f68_95ba_50376d52e32a.slice/crio-23705808b5f6362c550e7f2d531e8ee96a81bd38772ae50d18188f1ba54d0cdf WatchSource:0}: Error finding container 23705808b5f6362c550e7f2d531e8ee96a81bd38772ae50d18188f1ba54d0cdf: Status 404 returned error can't find the container with id 23705808b5f6362c550e7f2d531e8ee96a81bd38772ae50d18188f1ba54d0cdf
Sep 06 10:30:23.767862 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.767842    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-zmk6j\" (UID: \"4e55abd8-3289-4fff-8d8f-34129da7fb69\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"
Sep 06 10:30:23.767973 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.767957    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:23.768018 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.767998    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 06 10:30:23.768057 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.768047    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls podName:4e55abd8-3289-4fff-8d8f-34129da7fb69 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:24.768032885 +0000 UTC m=+37.217813647 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-zmk6j" (UID: "4e55abd8-3289-4fff-8d8f-34129da7fb69") : secret "cluster-monitoring-operator-tls" not found
Sep 06 10:30:23.768134 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.768117    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 06 10:30:23.768172 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.768137    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b78f8f8f-f8nfj: secret "image-registry-tls" not found
Sep 06 10:30:23.768205 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.768187    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls podName:69d7eb68-0175-46cd-9464-19e90cdcc1a3 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:24.768170928 +0000 UTC m=+37.217951694 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls") pod "image-registry-7b78f8f8f-f8nfj" (UID: "69d7eb68-0175-46cd-9464-19e90cdcc1a3") : secret "image-registry-tls" not found
Sep 06 10:30:23.868999 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.868968    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-7jsrm\" (UID: \"c2cbcefd-af14-4314-a52c-674e6775c298\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm"
Sep 06 10:30:23.869115 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.869076    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 06 10:30:23.869163 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.869119    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls\") pod \"dns-default-dltkp\" (UID: \"ec03af21-0135-4ba1-b4be-06adf19748ed\") " pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:23.869163 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.869130    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls podName:c2cbcefd-af14-4314-a52c-674e6775c298 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:24.869117316 +0000 UTC m=+37.318898062 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-7jsrm" (UID: "c2cbcefd-af14-4314-a52c-674e6775c298") : secret "samples-operator-tls" not found
Sep 06 10:30:23.869235 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.869162    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:23.869235 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.869183    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 10:30:23.869235 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:23.869198    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:23.869235 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.869225    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls podName:ec03af21-0135-4ba1-b4be-06adf19748ed nodeName:}" failed. No retries permitted until 2026-09-06 10:30:24.869210274 +0000 UTC m=+37.318991038 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls") pod "dns-default-dltkp" (UID: "ec03af21-0135-4ba1-b4be-06adf19748ed") : secret "dns-default-metrics-tls" not found
Sep 06 10:30:23.869371 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.869253    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle podName:0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c nodeName:}" failed. No retries permitted until 2026-09-06 10:30:24.869245816 +0000 UTC m=+37.319026562 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle") pod "router-default-78f4854998-kj5tp" (UID: "0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c") : configmap references non-existent config key: service-ca.crt
Sep 06 10:30:23.869371 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.869267    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 06 10:30:23.869371 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:23.869314    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs podName:0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c nodeName:}" failed. No retries permitted until 2026-09-06 10:30:24.869302361 +0000 UTC m=+37.319083138 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs") pod "router-default-78f4854998-kj5tp" (UID: "0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c") : secret "router-metrics-certs-default" not found
Sep 06 10:30:24.278669 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:24.278594    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-9lt24" event={"ID":"938e293f-7414-4bdd-946f-0b463ddc8ecb","Type":"ContainerStarted","Data":"59ec49ff61a6767047735f07bb782de7b7b640fb3e04a64670bd1fbfb1fc6b5a"}
Sep 06 10:30:24.281672 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:24.281640    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7jr4k" event={"ID":"99fa6d2e-adc6-4bd8-b6b0-e73045c36405","Type":"ContainerStarted","Data":"389a6ca94336ee2568ca183e8fa26ded8c73274b80e7f5ff9a6a8657879e0d43"}
Sep 06 10:30:24.283726 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:24.283662    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-74t29" event={"ID":"0f62f367-e8fd-4f68-95ba-50376d52e32a","Type":"ContainerStarted","Data":"23705808b5f6362c550e7f2d531e8ee96a81bd38772ae50d18188f1ba54d0cdf"}
Sep 06 10:30:24.286396 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:24.286366    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-gmxdr" event={"ID":"a44ce5d7-ab56-4b70-9f5f-8b312a5bb8ac","Type":"ContainerStarted","Data":"11896d8833fa0a8cc0b0c0d120bf00ea294f9e855b4c0b158bb328b60b205533"}
Sep 06 10:30:24.288093 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:24.288019    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g" event={"ID":"679d9837-653d-4ac4-b786-dd55ac8a5e06","Type":"ContainerStarted","Data":"0686f2b8252de9f395efd0d36e609db7ba670c73c4d8b35d72d2ae067157e1d6"}
Sep 06 10:30:24.290064 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:24.290015    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-p8s2s" event={"ID":"1462951b-ed57-4e82-8a1d-5d5f08f73ee9","Type":"ContainerStarted","Data":"4c2044a1296950e03708d900e716ece79ccfa8f89fdcacc1db7cb41f1a41bbdb"}
Sep 06 10:30:24.293814 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:24.293764    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-tr4gb" event={"ID":"9dc8c2c3-d63b-4ae4-8458-e37779164ff2","Type":"ContainerStarted","Data":"afe404942f12bd071dae3b71c2983cfae25df5c449cde2b5f2c039a1b58b4fa2"}
Sep 06 10:30:24.780138 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:24.778056    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:24.780138 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:24.778365    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-zmk6j\" (UID: \"4e55abd8-3289-4fff-8d8f-34129da7fb69\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"
Sep 06 10:30:24.780138 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:24.779643    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 06 10:30:24.780138 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:24.779663    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b78f8f8f-f8nfj: secret "image-registry-tls" not found
Sep 06 10:30:24.780138 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:24.779736    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls podName:69d7eb68-0175-46cd-9464-19e90cdcc1a3 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:26.77971639 +0000 UTC m=+39.229497153 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls") pod "image-registry-7b78f8f8f-f8nfj" (UID: "69d7eb68-0175-46cd-9464-19e90cdcc1a3") : secret "image-registry-tls" not found
Sep 06 10:30:24.787720 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:24.780675    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 06 10:30:24.787720 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:24.780734    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls podName:4e55abd8-3289-4fff-8d8f-34129da7fb69 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:26.780718194 +0000 UTC m=+39.230498957 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-zmk6j" (UID: "4e55abd8-3289-4fff-8d8f-34129da7fb69") : secret "cluster-monitoring-operator-tls" not found
Sep 06 10:30:24.880942 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:24.879774    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:24.880942 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:24.879886    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-7jsrm\" (UID: \"c2cbcefd-af14-4314-a52c-674e6775c298\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm"
Sep 06 10:30:24.880942 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:24.879970    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls\") pod \"dns-default-dltkp\" (UID: \"ec03af21-0135-4ba1-b4be-06adf19748ed\") " pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:24.880942 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:24.880009    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:24.880942 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:24.880155    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 06 10:30:24.880942 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:24.880263    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs podName:0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c nodeName:}" failed. No retries permitted until 2026-09-06 10:30:26.880243535 +0000 UTC m=+39.330024303 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs") pod "router-default-78f4854998-kj5tp" (UID: "0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c") : secret "router-metrics-certs-default" not found
Sep 06 10:30:24.880942 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:24.880686    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle podName:0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c nodeName:}" failed. No retries permitted until 2026-09-06 10:30:26.880669108 +0000 UTC m=+39.330449872 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle") pod "router-default-78f4854998-kj5tp" (UID: "0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c") : configmap references non-existent config key: service-ca.crt
Sep 06 10:30:24.880942 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:24.880763    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 06 10:30:24.880942 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:24.880802    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls podName:c2cbcefd-af14-4314-a52c-674e6775c298 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:26.880789512 +0000 UTC m=+39.330570263 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-7jsrm" (UID: "c2cbcefd-af14-4314-a52c-674e6775c298") : secret "samples-operator-tls" not found
Sep 06 10:30:24.880942 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:24.880862    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 10:30:24.880942 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:24.880891    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls podName:ec03af21-0135-4ba1-b4be-06adf19748ed nodeName:}" failed. No retries permitted until 2026-09-06 10:30:26.880880918 +0000 UTC m=+39.330661668 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls") pod "dns-default-dltkp" (UID: "ec03af21-0135-4ba1-b4be-06adf19748ed") : secret "dns-default-metrics-tls" not found
Sep 06 10:30:25.096537 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:25.096275    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-qh488"]
Sep 06 10:30:25.104515 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:25.104200    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-qh488"
Sep 06 10:30:25.109255 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:25.108838    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 06 10:30:25.109255 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:25.109082    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 06 10:30:25.109255 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:25.109220    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-lrlxd\""
Sep 06 10:30:25.112779 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:25.112730    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-qh488"]
Sep 06 10:30:25.183027 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:25.182987    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2b5a6c6a-2f52-4d65-8000-c39f44942495-nginx-conf\") pod \"networking-console-plugin-79977f9844-qh488\" (UID: \"2b5a6c6a-2f52-4d65-8000-c39f44942495\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qh488"
Sep 06 10:30:25.183225 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:25.183200    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qh488\" (UID: \"2b5a6c6a-2f52-4d65-8000-c39f44942495\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qh488"
Sep 06 10:30:25.221712 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:25.221664    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kpqgt_1fdbdca8-792e-4abe-8058-3cc6df71bedd/konnectivity-agent/0.log" limit=1024
Sep 06 10:30:25.285284 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:25.284795    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qh488\" (UID: \"2b5a6c6a-2f52-4d65-8000-c39f44942495\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qh488"
Sep 06 10:30:25.285284 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:25.284882    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2b5a6c6a-2f52-4d65-8000-c39f44942495-nginx-conf\") pod \"networking-console-plugin-79977f9844-qh488\" (UID: \"2b5a6c6a-2f52-4d65-8000-c39f44942495\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qh488"
Sep 06 10:30:25.285284 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:25.285117    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 06 10:30:25.285284 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:25.285189    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert podName:2b5a6c6a-2f52-4d65-8000-c39f44942495 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:25.785170589 +0000 UTC m=+38.234951361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-qh488" (UID: "2b5a6c6a-2f52-4d65-8000-c39f44942495") : secret "networking-console-plugin-cert" not found
Sep 06 10:30:25.285948 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:25.285600    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2b5a6c6a-2f52-4d65-8000-c39f44942495-nginx-conf\") pod \"networking-console-plugin-79977f9844-qh488\" (UID: \"2b5a6c6a-2f52-4d65-8000-c39f44942495\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qh488"
Sep 06 10:30:25.320836 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:25.320783    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kpqgt_1fdbdca8-792e-4abe-8058-3cc6df71bedd/konnectivity-agent/0.log" limit=1024
Sep 06 10:30:25.423345 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:25.423270    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kpqgt_1fdbdca8-792e-4abe-8058-3cc6df71bedd/konnectivity-agent/0.log" limit=1024
Sep 06 10:30:25.484679 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:25.484642    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kpqgt_1fdbdca8-792e-4abe-8058-3cc6df71bedd/konnectivity-agent/0.log" limit=1024
Sep 06 10:30:25.790978 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:25.790884    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qh488\" (UID: \"2b5a6c6a-2f52-4d65-8000-c39f44942495\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qh488"
Sep 06 10:30:25.791242 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:25.791153    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 06 10:30:25.791346 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:25.791249    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert podName:2b5a6c6a-2f52-4d65-8000-c39f44942495 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:26.791227073 +0000 UTC m=+39.241007820 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-qh488" (UID: "2b5a6c6a-2f52-4d65-8000-c39f44942495") : secret "networking-console-plugin-cert" not found
Sep 06 10:30:26.801287 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:26.801247    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qh488\" (UID: \"2b5a6c6a-2f52-4d65-8000-c39f44942495\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qh488"
Sep 06 10:30:26.801708 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:26.801413    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:26.801708 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:26.801515    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-zmk6j\" (UID: \"4e55abd8-3289-4fff-8d8f-34129da7fb69\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"
Sep 06 10:30:26.801708 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:26.801416    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 06 10:30:26.801708 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:26.801609    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert podName:2b5a6c6a-2f52-4d65-8000-c39f44942495 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:28.801588051 +0000 UTC m=+41.251368801 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-qh488" (UID: "2b5a6c6a-2f52-4d65-8000-c39f44942495") : secret "networking-console-plugin-cert" not found
Sep 06 10:30:26.801708 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:26.801531    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 06 10:30:26.801708 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:26.801634    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 06 10:30:26.801708 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:26.801685    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls podName:4e55abd8-3289-4fff-8d8f-34129da7fb69 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:30.801670368 +0000 UTC m=+43.251451127 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-zmk6j" (UID: "4e55abd8-3289-4fff-8d8f-34129da7fb69") : secret "cluster-monitoring-operator-tls" not found
Sep 06 10:30:26.801708 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:26.801635    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b78f8f8f-f8nfj: secret "image-registry-tls" not found
Sep 06 10:30:26.802104 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:26.801728    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls podName:69d7eb68-0175-46cd-9464-19e90cdcc1a3 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:30.801717083 +0000 UTC m=+43.251497832 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls") pod "image-registry-7b78f8f8f-f8nfj" (UID: "69d7eb68-0175-46cd-9464-19e90cdcc1a3") : secret "image-registry-tls" not found
Sep 06 10:30:26.902621 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:26.902582    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:26.902771 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:26.902684    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-7jsrm\" (UID: \"c2cbcefd-af14-4314-a52c-674e6775c298\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm"
Sep 06 10:30:26.902771 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:26.902761    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle podName:0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c nodeName:}" failed. No retries permitted until 2026-09-06 10:30:30.902740521 +0000 UTC m=+43.352521282 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle") pod "router-default-78f4854998-kj5tp" (UID: "0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c") : configmap references non-existent config key: service-ca.crt
Sep 06 10:30:26.902907 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:26.902811    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 06 10:30:26.902907 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:26.902859    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls podName:c2cbcefd-af14-4314-a52c-674e6775c298 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:30.902847666 +0000 UTC m=+43.352628433 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-7jsrm" (UID: "c2cbcefd-af14-4314-a52c-674e6775c298") : secret "samples-operator-tls" not found
Sep 06 10:30:26.902907 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:26.902807    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls\") pod \"dns-default-dltkp\" (UID: \"ec03af21-0135-4ba1-b4be-06adf19748ed\") " pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:26.902907 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:26.902876    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 10:30:26.903083 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:26.902916    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls podName:ec03af21-0135-4ba1-b4be-06adf19748ed nodeName:}" failed. No retries permitted until 2026-09-06 10:30:30.902903508 +0000 UTC m=+43.352684268 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls") pod "dns-default-dltkp" (UID: "ec03af21-0135-4ba1-b4be-06adf19748ed") : secret "dns-default-metrics-tls" not found
Sep 06 10:30:26.903083 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:26.902908    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:26.903083 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:26.903014    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 06 10:30:26.903083 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:26.903064    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs podName:0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c nodeName:}" failed. No retries permitted until 2026-09-06 10:30:30.903051891 +0000 UTC m=+43.352832643 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs") pod "router-default-78f4854998-kj5tp" (UID: "0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c") : secret "router-metrics-certs-default" not found
Sep 06 10:30:27.120433 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:27.120370    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kpqgt_1fdbdca8-792e-4abe-8058-3cc6df71bedd/konnectivity-agent/0.log" limit=1024
Sep 06 10:30:27.221153 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:27.221117    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kpqgt_1fdbdca8-792e-4abe-8058-3cc6df71bedd/konnectivity-agent/0.log" limit=1024
Sep 06 10:30:28.820916 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:28.820880    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qh488\" (UID: \"2b5a6c6a-2f52-4d65-8000-c39f44942495\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qh488"
Sep 06 10:30:28.821262 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:28.821037    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 06 10:30:28.821262 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:28.821092    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert podName:2b5a6c6a-2f52-4d65-8000-c39f44942495 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:32.821079584 +0000 UTC m=+45.270860329 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-qh488" (UID: "2b5a6c6a-2f52-4d65-8000-c39f44942495") : secret "networking-console-plugin-cert" not found
Sep 06 10:30:30.838014 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:30.837974    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:30.838454 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:30.838087    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-zmk6j\" (UID: \"4e55abd8-3289-4fff-8d8f-34129da7fb69\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"
Sep 06 10:30:30.838454 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:30.838146    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 06 10:30:30.838454 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:30.838176    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b78f8f8f-f8nfj: secret "image-registry-tls" not found
Sep 06 10:30:30.838454 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:30.838250    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls podName:69d7eb68-0175-46cd-9464-19e90cdcc1a3 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:38.838229525 +0000 UTC m=+51.288010292 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls") pod "image-registry-7b78f8f8f-f8nfj" (UID: "69d7eb68-0175-46cd-9464-19e90cdcc1a3") : secret "image-registry-tls" not found
Sep 06 10:30:30.838454 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:30.838276    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 06 10:30:30.838454 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:30.838348    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls podName:4e55abd8-3289-4fff-8d8f-34129da7fb69 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:38.838330604 +0000 UTC m=+51.288111370 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-zmk6j" (UID: "4e55abd8-3289-4fff-8d8f-34129da7fb69") : secret "cluster-monitoring-operator-tls" not found
Sep 06 10:30:30.938752 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:30.938713    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls\") pod \"dns-default-dltkp\" (UID: \"ec03af21-0135-4ba1-b4be-06adf19748ed\") " pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:30.938922 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:30.938765    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:30.938922 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:30.938802    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:30.938922 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:30.938857    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 10:30:30.938922 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:30.938881    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-7jsrm\" (UID: \"c2cbcefd-af14-4314-a52c-674e6775c298\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm"
Sep 06 10:30:30.938922 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:30.938914    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls podName:ec03af21-0135-4ba1-b4be-06adf19748ed nodeName:}" failed. No retries permitted until 2026-09-06 10:30:38.938898932 +0000 UTC m=+51.388679677 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls") pod "dns-default-dltkp" (UID: "ec03af21-0135-4ba1-b4be-06adf19748ed") : secret "dns-default-metrics-tls" not found
Sep 06 10:30:30.939121 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:30.938919    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 06 10:30:30.939121 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:30.938966    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 06 10:30:30.939121 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:30.938981    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs podName:0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c nodeName:}" failed. No retries permitted until 2026-09-06 10:30:38.93896452 +0000 UTC m=+51.388745283 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs") pod "router-default-78f4854998-kj5tp" (UID: "0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c") : secret "router-metrics-certs-default" not found
Sep 06 10:30:30.939121 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:30.938995    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle podName:0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c nodeName:}" failed. No retries permitted until 2026-09-06 10:30:38.938989183 +0000 UTC m=+51.388769929 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle") pod "router-default-78f4854998-kj5tp" (UID: "0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c") : configmap references non-existent config key: service-ca.crt
Sep 06 10:30:30.939121 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:30.939006    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls podName:c2cbcefd-af14-4314-a52c-674e6775c298 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:38.938999924 +0000 UTC m=+51.388780670 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-7jsrm" (UID: "c2cbcefd-af14-4314-a52c-674e6775c298") : secret "samples-operator-tls" not found
Sep 06 10:30:32.146281 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.146245    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl"]
Sep 06 10:30:32.167301 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.167276    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl"]
Sep 06 10:30:32.167448 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.167377    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl"
Sep 06 10:30:32.170603 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.170548    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 06 10:30:32.170705 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.170652    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bf9d9b777-fwthv"]
Sep 06 10:30:32.170866 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.170849    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 06 10:30:32.171067 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.171053    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-f9c9d\""
Sep 06 10:30:32.171227 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.171214    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 06 10:30:32.171405 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.171392    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 06 10:30:32.183805 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.183784    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"]
Sep 06 10:30:32.184321 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.183997    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bf9d9b777-fwthv"
Sep 06 10:30:32.192540 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.189126    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 06 10:30:32.212366 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.212345    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bf9d9b777-fwthv"]
Sep 06 10:30:32.212461 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.212392    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"]
Sep 06 10:30:32.212522 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.212478    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.214708 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.214691    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 06 10:30:32.214847 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.214709    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 06 10:30:32.215307 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.215290    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 06 10:30:32.215424 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.215404    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 06 10:30:32.250369 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.250344    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/17a46f13-f538-4f23-90b8-654975a915d8-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl\" (UID: \"17a46f13-f538-4f23-90b8-654975a915d8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl"
Sep 06 10:30:32.250560 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.250381    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smrx6\" (UniqueName: \"kubernetes.io/projected/eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec-kube-api-access-smrx6\") pod \"klusterlet-addon-workmgr-6bf9d9b777-fwthv\" (UID: \"eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bf9d9b777-fwthv"
Sep 06 10:30:32.250560 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.250413    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7c24bfc5-c910-4ef3-9d93-d27221a83328-ca\") pod \"cluster-proxy-proxy-agent-6c95667689-qnzwl\" (UID: \"7c24bfc5-c910-4ef3-9d93-d27221a83328\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.250560 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.250437    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c24bfc5-c910-4ef3-9d93-d27221a83328-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c95667689-qnzwl\" (UID: \"7c24bfc5-c910-4ef3-9d93-d27221a83328\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.250560 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.250541    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvtcc\" (UniqueName: \"kubernetes.io/projected/17a46f13-f538-4f23-90b8-654975a915d8-kube-api-access-bvtcc\") pod \"managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl\" (UID: \"17a46f13-f538-4f23-90b8-654975a915d8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl"
Sep 06 10:30:32.250775 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.250576    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7c24bfc5-c910-4ef3-9d93-d27221a83328-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c95667689-qnzwl\" (UID: \"7c24bfc5-c910-4ef3-9d93-d27221a83328\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.250775 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.250605    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7c24bfc5-c910-4ef3-9d93-d27221a83328-hub\") pod \"cluster-proxy-proxy-agent-6c95667689-qnzwl\" (UID: \"7c24bfc5-c910-4ef3-9d93-d27221a83328\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.250775 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.250625    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec-tmp\") pod \"klusterlet-addon-workmgr-6bf9d9b777-fwthv\" (UID: \"eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bf9d9b777-fwthv"
Sep 06 10:30:32.250775 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.250709    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7c24bfc5-c910-4ef3-9d93-d27221a83328-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c95667689-qnzwl\" (UID: \"7c24bfc5-c910-4ef3-9d93-d27221a83328\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.250775 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.250772    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec-klusterlet-config\") pod \"klusterlet-addon-workmgr-6bf9d9b777-fwthv\" (UID: \"eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bf9d9b777-fwthv"
Sep 06 10:30:32.251015 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.250823    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mztbg\" (UniqueName: \"kubernetes.io/projected/7c24bfc5-c910-4ef3-9d93-d27221a83328-kube-api-access-mztbg\") pod \"cluster-proxy-proxy-agent-6c95667689-qnzwl\" (UID: \"7c24bfc5-c910-4ef3-9d93-d27221a83328\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.319396 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.319363    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-74t29" event={"ID":"0f62f367-e8fd-4f68-95ba-50376d52e32a","Type":"ContainerStarted","Data":"0651fd053ee25c7aee4aa050634effe4f8e4699b8452693b7b1fc1c2dfce90ee"}
Sep 06 10:30:32.321153 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.321130    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-gmxdr" event={"ID":"a44ce5d7-ab56-4b70-9f5f-8b312a5bb8ac","Type":"ContainerStarted","Data":"81f54ea3da4bf338d6cd0ebd010f584ec1979f339224200c79d8a8ac6a0ff804"}
Sep 06 10:30:32.323334 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.323116    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g" event={"ID":"679d9837-653d-4ac4-b786-dd55ac8a5e06","Type":"ContainerStarted","Data":"a9ac8581937621eb7e28a4ef5ae74e120766ab4e2eca36bb9b85bb927a5f7ce1"}
Sep 06 10:30:32.324824 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.324795    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-p8s2s" event={"ID":"1462951b-ed57-4e82-8a1d-5d5f08f73ee9","Type":"ContainerStarted","Data":"71f6f9b0e8bac2e629abba0ab81ba7470d7a8ee0dcbde3d1f0c876a1841cb32b"}
Sep 06 10:30:32.326446 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.326423    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-tr4gb" event={"ID":"9dc8c2c3-d63b-4ae4-8458-e37779164ff2","Type":"ContainerStarted","Data":"40cd7aa976d544544d03c6fb7b45661dda064c9c10f3eb31fa6b80fecda1b87e"}
Sep 06 10:30:32.328557 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.328537    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/0.log"
Sep 06 10:30:32.328638 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.328575    2557 generic.go:356] "Generic (PLEG): container finished" podID="938e293f-7414-4bdd-946f-0b463ddc8ecb" containerID="56bc24b507d58538b3dcc024808b8abdfc2e2f84e6378142b418fe24b9c83b07" exitCode=255
Sep 06 10:30:32.328688 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.328655    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-9lt24" event={"ID":"938e293f-7414-4bdd-946f-0b463ddc8ecb","Type":"ContainerDied","Data":"56bc24b507d58538b3dcc024808b8abdfc2e2f84e6378142b418fe24b9c83b07"}
Sep 06 10:30:32.328856 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.328838    2557 scope.go:117] "RemoveContainer" containerID="56bc24b507d58538b3dcc024808b8abdfc2e2f84e6378142b418fe24b9c83b07"
Sep 06 10:30:32.330886 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.330845    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7jr4k" event={"ID":"99fa6d2e-adc6-4bd8-b6b0-e73045c36405","Type":"ContainerStarted","Data":"c1e930b360ff0c5a1a0f732db97ae9c3f8e8011c178d22bb23fc7cc42175a87f"}
Sep 06 10:30:32.335124 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.335074    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-74t29" podStartSLOduration=2.660203615 podStartE2EDuration="10.335057507s" totalImagesPullingTime="7.674853892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:32.33389349 +0000 UTC m=+44.783674259" watchObservedRunningTime="2026-09-06 10:30:32.335057507 +0000 UTC m=+44.784838271"
Sep 06 10:30:32.351259 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.351189    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g" podStartSLOduration=5.613367881 podStartE2EDuration="13.351173022s" totalImagesPullingTime="7.737805141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:32.350462245 +0000 UTC m=+44.800243016" watchObservedRunningTime="2026-09-06 10:30:32.351173022 +0000 UTC m=+44.800953787"
Sep 06 10:30:32.351444 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.351424    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvtcc\" (UniqueName: \"kubernetes.io/projected/17a46f13-f538-4f23-90b8-654975a915d8-kube-api-access-bvtcc\") pod \"managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl\" (UID: \"17a46f13-f538-4f23-90b8-654975a915d8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl"
Sep 06 10:30:32.351634 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.351620    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7c24bfc5-c910-4ef3-9d93-d27221a83328-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c95667689-qnzwl\" (UID: \"7c24bfc5-c910-4ef3-9d93-d27221a83328\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.351797 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.351783    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7c24bfc5-c910-4ef3-9d93-d27221a83328-hub\") pod \"cluster-proxy-proxy-agent-6c95667689-qnzwl\" (UID: \"7c24bfc5-c910-4ef3-9d93-d27221a83328\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.355549 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.353040    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec-tmp\") pod \"klusterlet-addon-workmgr-6bf9d9b777-fwthv\" (UID: \"eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bf9d9b777-fwthv"
Sep 06 10:30:32.355549 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.353140    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7c24bfc5-c910-4ef3-9d93-d27221a83328-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c95667689-qnzwl\" (UID: \"7c24bfc5-c910-4ef3-9d93-d27221a83328\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.355549 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.353185    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec-klusterlet-config\") pod \"klusterlet-addon-workmgr-6bf9d9b777-fwthv\" (UID: \"eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bf9d9b777-fwthv"
Sep 06 10:30:32.355549 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.353268    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mztbg\" (UniqueName: \"kubernetes.io/projected/7c24bfc5-c910-4ef3-9d93-d27221a83328-kube-api-access-mztbg\") pod \"cluster-proxy-proxy-agent-6c95667689-qnzwl\" (UID: \"7c24bfc5-c910-4ef3-9d93-d27221a83328\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.355549 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.353384    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/17a46f13-f538-4f23-90b8-654975a915d8-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl\" (UID: \"17a46f13-f538-4f23-90b8-654975a915d8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl"
Sep 06 10:30:32.355549 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.353408    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smrx6\" (UniqueName: \"kubernetes.io/projected/eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec-kube-api-access-smrx6\") pod \"klusterlet-addon-workmgr-6bf9d9b777-fwthv\" (UID: \"eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bf9d9b777-fwthv"
Sep 06 10:30:32.355549 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.353458    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7c24bfc5-c910-4ef3-9d93-d27221a83328-ca\") pod \"cluster-proxy-proxy-agent-6c95667689-qnzwl\" (UID: \"7c24bfc5-c910-4ef3-9d93-d27221a83328\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.355549 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.353481    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c24bfc5-c910-4ef3-9d93-d27221a83328-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c95667689-qnzwl\" (UID: \"7c24bfc5-c910-4ef3-9d93-d27221a83328\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.355549 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.353540    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec-tmp\") pod \"klusterlet-addon-workmgr-6bf9d9b777-fwthv\" (UID: \"eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bf9d9b777-fwthv"
Sep 06 10:30:32.356994 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.356892    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c24bfc5-c910-4ef3-9d93-d27221a83328-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c95667689-qnzwl\" (UID: \"7c24bfc5-c910-4ef3-9d93-d27221a83328\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.369919 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.368689    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7c24bfc5-c910-4ef3-9d93-d27221a83328-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c95667689-qnzwl\" (UID: \"7c24bfc5-c910-4ef3-9d93-d27221a83328\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.369919 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.368749    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7c24bfc5-c910-4ef3-9d93-d27221a83328-hub\") pod \"cluster-proxy-proxy-agent-6c95667689-qnzwl\" (UID: \"7c24bfc5-c910-4ef3-9d93-d27221a83328\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.369919 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.369051    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec-klusterlet-config\") pod \"klusterlet-addon-workmgr-6bf9d9b777-fwthv\" (UID: \"eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bf9d9b777-fwthv"
Sep 06 10:30:32.369919 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.369072    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7c24bfc5-c910-4ef3-9d93-d27221a83328-ca\") pod \"cluster-proxy-proxy-agent-6c95667689-qnzwl\" (UID: \"7c24bfc5-c910-4ef3-9d93-d27221a83328\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.369919 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.369202    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/17a46f13-f538-4f23-90b8-654975a915d8-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl\" (UID: \"17a46f13-f538-4f23-90b8-654975a915d8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl"
Sep 06 10:30:32.369919 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.369545    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7c24bfc5-c910-4ef3-9d93-d27221a83328-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c95667689-qnzwl\" (UID: \"7c24bfc5-c910-4ef3-9d93-d27221a83328\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.371510 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.371299    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-tr4gb" podStartSLOduration=5.610251192 podStartE2EDuration="13.371281804s" totalImagesPullingTime="7.761030612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:32.369909554 +0000 UTC m=+44.819690324" watchObservedRunningTime="2026-09-06 10:30:32.371281804 +0000 UTC m=+44.821062569"
Sep 06 10:30:32.373099 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.372416    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvtcc\" (UniqueName: \"kubernetes.io/projected/17a46f13-f538-4f23-90b8-654975a915d8-kube-api-access-bvtcc\") pod \"managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl\" (UID: \"17a46f13-f538-4f23-90b8-654975a915d8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl"
Sep 06 10:30:32.373334 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.373312    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smrx6\" (UniqueName: \"kubernetes.io/projected/eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec-kube-api-access-smrx6\") pod \"klusterlet-addon-workmgr-6bf9d9b777-fwthv\" (UID: \"eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bf9d9b777-fwthv"
Sep 06 10:30:32.374760 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.374722    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mztbg\" (UniqueName: \"kubernetes.io/projected/7c24bfc5-c910-4ef3-9d93-d27221a83328-kube-api-access-mztbg\") pod \"cluster-proxy-proxy-agent-6c95667689-qnzwl\" (UID: \"7c24bfc5-c910-4ef3-9d93-d27221a83328\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.402033 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.401984    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-gmxdr" podStartSLOduration=5.673517252 podStartE2EDuration="13.401971695s" totalImagesPullingTime="7.728454443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:32.400984019 +0000 UTC m=+44.850764789" watchObservedRunningTime="2026-09-06 10:30:32.401971695 +0000 UTC m=+44.851752463"
Sep 06 10:30:32.417857 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.417810    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7jr4k" podStartSLOduration=5.561463183 podStartE2EDuration="13.417792428s" totalImagesPullingTime="7.856329245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:32.416258297 +0000 UTC m=+44.866039065" watchObservedRunningTime="2026-09-06 10:30:32.417792428 +0000 UTC m=+44.867573197"
Sep 06 10:30:32.433081 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.432724    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-p8s2s" podStartSLOduration=5.628525705 podStartE2EDuration="13.4327069s" totalImagesPullingTime="7.804181195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:32.432041853 +0000 UTC m=+44.881822621" watchObservedRunningTime="2026-09-06 10:30:32.4327069 +0000 UTC m=+44.882487669"
Sep 06 10:30:32.490472 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.490441    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl"
Sep 06 10:30:32.503601 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.503573    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bf9d9b777-fwthv"
Sep 06 10:30:32.522613 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.522542    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:30:32.791643 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.791613    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl"]
Sep 06 10:30:32.795229 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:32.795199    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17a46f13_f538_4f23_90b8_654975a915d8.slice/crio-bfaf32d7ae3962ba68a3b3bbc60c963f3e8ff019e22e818353d23286d8639da5 WatchSource:0}: Error finding container bfaf32d7ae3962ba68a3b3bbc60c963f3e8ff019e22e818353d23286d8639da5: Status 404 returned error can't find the container with id bfaf32d7ae3962ba68a3b3bbc60c963f3e8ff019e22e818353d23286d8639da5
Sep 06 10:30:32.847057 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.847035    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bf9d9b777-fwthv"]
Sep 06 10:30:32.849385 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:32.849361    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb2d3f1b_bdba_4c1d_a75f_9dbefc69f9ec.slice/crio-f38ce8a98ac8f58b5d4c054f4377ff4d93e7b013c9110be344a42c4490bd78bf WatchSource:0}: Error finding container f38ce8a98ac8f58b5d4c054f4377ff4d93e7b013c9110be344a42c4490bd78bf: Status 404 returned error can't find the container with id f38ce8a98ac8f58b5d4c054f4377ff4d93e7b013c9110be344a42c4490bd78bf
Sep 06 10:30:32.859522 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.859477    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qh488\" (UID: \"2b5a6c6a-2f52-4d65-8000-c39f44942495\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qh488"
Sep 06 10:30:32.859621 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:32.859603    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 06 10:30:32.859686 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:32.859673    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert podName:2b5a6c6a-2f52-4d65-8000-c39f44942495 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:40.859653885 +0000 UTC m=+53.309434635 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-qh488" (UID: "2b5a6c6a-2f52-4d65-8000-c39f44942495") : secret "networking-console-plugin-cert" not found
Sep 06 10:30:32.872558 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:32.872540    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"]
Sep 06 10:30:32.874750 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:32.874726    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c24bfc5_c910_4ef3_9d93_d27221a83328.slice/crio-5904302aa793574a8559ebfad4226d752d5019dd5564aa9bfb853ca942b019fa WatchSource:0}: Error finding container 5904302aa793574a8559ebfad4226d752d5019dd5564aa9bfb853ca942b019fa: Status 404 returned error can't find the container with id 5904302aa793574a8559ebfad4226d752d5019dd5564aa9bfb853ca942b019fa
Sep 06 10:30:33.020194 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.020096    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-k9kzz"]
Sep 06 10:30:33.048537 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.048510    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-k9kzz"]
Sep 06 10:30:33.048706 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.048583    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-k9kzz"
Sep 06 10:30:33.051613 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.051585    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 06 10:30:33.051837 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.051798    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 06 10:30:33.052399 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.052376    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-bhs87\""
Sep 06 10:30:33.167944 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.167755    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drc8n\" (UniqueName: \"kubernetes.io/projected/840d1c17-0109-4541-93cb-7409823e8f70-kube-api-access-drc8n\") pod \"migrator-fdbb8fdff-k9kzz\" (UID: \"840d1c17-0109-4541-93cb-7409823e8f70\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-k9kzz"
Sep 06 10:30:33.268902 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.268873    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drc8n\" (UniqueName: \"kubernetes.io/projected/840d1c17-0109-4541-93cb-7409823e8f70-kube-api-access-drc8n\") pod \"migrator-fdbb8fdff-k9kzz\" (UID: \"840d1c17-0109-4541-93cb-7409823e8f70\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-k9kzz"
Sep 06 10:30:33.279774 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.279722    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drc8n\" (UniqueName: \"kubernetes.io/projected/840d1c17-0109-4541-93cb-7409823e8f70-kube-api-access-drc8n\") pod \"migrator-fdbb8fdff-k9kzz\" (UID: \"840d1c17-0109-4541-93cb-7409823e8f70\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-k9kzz"
Sep 06 10:30:33.335895 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.335858    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl" event={"ID":"17a46f13-f538-4f23-90b8-654975a915d8","Type":"ContainerStarted","Data":"bfaf32d7ae3962ba68a3b3bbc60c963f3e8ff019e22e818353d23286d8639da5"}
Sep 06 10:30:33.337012 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.336988    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl" event={"ID":"7c24bfc5-c910-4ef3-9d93-d27221a83328","Type":"ContainerStarted","Data":"5904302aa793574a8559ebfad4226d752d5019dd5564aa9bfb853ca942b019fa"}
Sep 06 10:30:33.338095 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.338071    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bf9d9b777-fwthv" event={"ID":"eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec","Type":"ContainerStarted","Data":"f38ce8a98ac8f58b5d4c054f4377ff4d93e7b013c9110be344a42c4490bd78bf"}
Sep 06 10:30:33.339654 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.339637    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/1.log"
Sep 06 10:30:33.340035 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.340019    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/0.log"
Sep 06 10:30:33.340122 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.340048    2557 generic.go:356] "Generic (PLEG): container finished" podID="938e293f-7414-4bdd-946f-0b463ddc8ecb" containerID="f48248402720e700b261aaf78a26f909b7529dce6e6a3408289dd6613214d744" exitCode=255
Sep 06 10:30:33.340210 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.340156    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-9lt24" event={"ID":"938e293f-7414-4bdd-946f-0b463ddc8ecb","Type":"ContainerDied","Data":"f48248402720e700b261aaf78a26f909b7529dce6e6a3408289dd6613214d744"}
Sep 06 10:30:33.340268 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.340229    2557 scope.go:117] "RemoveContainer" containerID="56bc24b507d58538b3dcc024808b8abdfc2e2f84e6378142b418fe24b9c83b07"
Sep 06 10:30:33.340372 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.340352    2557 scope.go:117] "RemoveContainer" containerID="f48248402720e700b261aaf78a26f909b7529dce6e6a3408289dd6613214d744"
Sep 06 10:30:33.340595 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:33.340566    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-9lt24_openshift-console-operator(938e293f-7414-4bdd-946f-0b463ddc8ecb)\"" pod="openshift-console-operator/console-operator-85b454874b-9lt24" podUID="938e293f-7414-4bdd-946f-0b463ddc8ecb"
Sep 06 10:30:33.359665 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.359646    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-k9kzz"
Sep 06 10:30:33.406189 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.406165    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:30:33.406283 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.406212    2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:30:33.565874 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.565552    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pfxj6"]
Sep 06 10:30:33.584957 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.584761    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pfxj6"]
Sep 06 10:30:33.585140 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.585118    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:33.588342 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.587291    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 06 10:30:33.588342 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.587509    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-l4ftj\""
Sep 06 10:30:33.588342 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.587552    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 06 10:30:33.661636 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.661406    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-k9kzz"]
Sep 06 10:30:33.664285 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:33.664257    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod840d1c17_0109_4541_93cb_7409823e8f70.slice/crio-f4266c99a1cfca79799288cbbd6e12fa5483abfea6b053f2b8d5d593ea5f554a WatchSource:0}: Error finding container f4266c99a1cfca79799288cbbd6e12fa5483abfea6b053f2b8d5d593ea5f554a: Status 404 returned error can't find the container with id f4266c99a1cfca79799288cbbd6e12fa5483abfea6b053f2b8d5d593ea5f554a
Sep 06 10:30:33.673047 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.673022    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t52xt\" (UniqueName: \"kubernetes.io/projected/dace2beb-5038-4b5b-936c-120b52c07ed4-kube-api-access-t52xt\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:33.673241 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.673175    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dace2beb-5038-4b5b-936c-120b52c07ed4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:33.673375 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.673355    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dace2beb-5038-4b5b-936c-120b52c07ed4-crio-socket\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:33.673439 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.673389    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dace2beb-5038-4b5b-936c-120b52c07ed4-data-volume\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:33.673439 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.673417    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dace2beb-5038-4b5b-936c-120b52c07ed4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:33.774409 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.774371    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dace2beb-5038-4b5b-936c-120b52c07ed4-crio-socket\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:33.774409 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.774414    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dace2beb-5038-4b5b-936c-120b52c07ed4-data-volume\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:33.774644 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.774439    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dace2beb-5038-4b5b-936c-120b52c07ed4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:33.774644 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.774523    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t52xt\" (UniqueName: \"kubernetes.io/projected/dace2beb-5038-4b5b-936c-120b52c07ed4-kube-api-access-t52xt\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:33.774644 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.774625    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dace2beb-5038-4b5b-936c-120b52c07ed4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:33.774801 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:33.774735    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 10:30:33.774850 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:33.774820    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dace2beb-5038-4b5b-936c-120b52c07ed4-kube-rbac-proxy-cm podName:dace2beb-5038-4b5b-936c-120b52c07ed4 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:34.274801102 +0000 UTC m=+46.724581867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dace2beb-5038-4b5b-936c-120b52c07ed4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pfxj6" (UID: "dace2beb-5038-4b5b-936c-120b52c07ed4") : configmap "kube-rbac-proxy" not found
Sep 06 10:30:33.775284 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:33.775015    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 10:30:33.775284 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:33.775070    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dace2beb-5038-4b5b-936c-120b52c07ed4-insights-runtime-extractor-tls podName:dace2beb-5038-4b5b-936c-120b52c07ed4 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:34.275055252 +0000 UTC m=+46.724836011 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dace2beb-5038-4b5b-936c-120b52c07ed4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pfxj6" (UID: "dace2beb-5038-4b5b-936c-120b52c07ed4") : secret "insights-runtime-extractor-tls" not found
Sep 06 10:30:33.775284 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.775125    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dace2beb-5038-4b5b-936c-120b52c07ed4-crio-socket\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:33.775541 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.775376    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dace2beb-5038-4b5b-936c-120b52c07ed4-data-volume\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:33.787125 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:33.787100    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t52xt\" (UniqueName: \"kubernetes.io/projected/dace2beb-5038-4b5b-936c-120b52c07ed4-kube-api-access-t52xt\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:34.280986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:34.280446    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dace2beb-5038-4b5b-936c-120b52c07ed4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:34.280986 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:34.280579    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 10:30:34.280986 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:34.280660    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dace2beb-5038-4b5b-936c-120b52c07ed4-kube-rbac-proxy-cm podName:dace2beb-5038-4b5b-936c-120b52c07ed4 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:35.280638854 +0000 UTC m=+47.730419606 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dace2beb-5038-4b5b-936c-120b52c07ed4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pfxj6" (UID: "dace2beb-5038-4b5b-936c-120b52c07ed4") : configmap "kube-rbac-proxy" not found
Sep 06 10:30:34.280986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:34.280693    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dace2beb-5038-4b5b-936c-120b52c07ed4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:34.280986 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:34.280930    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 10:30:34.281714 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:34.281016    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dace2beb-5038-4b5b-936c-120b52c07ed4-insights-runtime-extractor-tls podName:dace2beb-5038-4b5b-936c-120b52c07ed4 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:35.280972028 +0000 UTC m=+47.730752778 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dace2beb-5038-4b5b-936c-120b52c07ed4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pfxj6" (UID: "dace2beb-5038-4b5b-936c-120b52c07ed4") : secret "insights-runtime-extractor-tls" not found
Sep 06 10:30:34.351117 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:34.350964    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/1.log"
Sep 06 10:30:34.351421 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:34.351401    2557 scope.go:117] "RemoveContainer" containerID="f48248402720e700b261aaf78a26f909b7529dce6e6a3408289dd6613214d744"
Sep 06 10:30:34.351633 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:34.351609    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-9lt24_openshift-console-operator(938e293f-7414-4bdd-946f-0b463ddc8ecb)\"" pod="openshift-console-operator/console-operator-85b454874b-9lt24" podUID="938e293f-7414-4bdd-946f-0b463ddc8ecb"
Sep 06 10:30:34.354413 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:34.354339    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-k9kzz" event={"ID":"840d1c17-0109-4541-93cb-7409823e8f70","Type":"ContainerStarted","Data":"f4266c99a1cfca79799288cbbd6e12fa5483abfea6b053f2b8d5d593ea5f554a"}
Sep 06 10:30:35.297008 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.296164    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dace2beb-5038-4b5b-936c-120b52c07ed4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:35.297008 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.296302    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dace2beb-5038-4b5b-936c-120b52c07ed4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:35.297008 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:35.296471    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 10:30:35.297008 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:35.296553    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dace2beb-5038-4b5b-936c-120b52c07ed4-insights-runtime-extractor-tls podName:dace2beb-5038-4b5b-936c-120b52c07ed4 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:37.296535304 +0000 UTC m=+49.746316055 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dace2beb-5038-4b5b-936c-120b52c07ed4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pfxj6" (UID: "dace2beb-5038-4b5b-936c-120b52c07ed4") : secret "insights-runtime-extractor-tls" not found
Sep 06 10:30:35.297008 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:35.296919    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 10:30:35.297008 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:35.296971    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dace2beb-5038-4b5b-936c-120b52c07ed4-kube-rbac-proxy-cm podName:dace2beb-5038-4b5b-936c-120b52c07ed4 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:37.296955332 +0000 UTC m=+49.746736101 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dace2beb-5038-4b5b-936c-120b52c07ed4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pfxj6" (UID: "dace2beb-5038-4b5b-936c-120b52c07ed4") : configmap "kube-rbac-proxy" not found
Sep 06 10:30:35.359751 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.359677    2557 scope.go:117] "RemoveContainer" containerID="f48248402720e700b261aaf78a26f909b7529dce6e6a3408289dd6613214d744"
Sep 06 10:30:35.360095 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:35.359924    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-9lt24_openshift-console-operator(938e293f-7414-4bdd-946f-0b463ddc8ecb)\"" pod="openshift-console-operator/console-operator-85b454874b-9lt24" podUID="938e293f-7414-4bdd-946f-0b463ddc8ecb"
Sep 06 10:30:35.405387 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.405355    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-2z5m9"]
Sep 06 10:30:35.424569 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.423435    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-2z5m9"]
Sep 06 10:30:35.424569 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.423559    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-2z5m9"
Sep 06 10:30:35.426745 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.426598    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 06 10:30:35.426745 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.426618    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 06 10:30:35.429906 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.428738    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 06 10:30:35.429906 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.429114    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 06 10:30:35.430068 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.429903    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-4r4p9\""
Sep 06 10:30:35.497814 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.497599    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3f2fa8a9-5005-4054-94ca-b68986b62750-signing-key\") pod \"service-ca-75d99c484-2z5m9\" (UID: \"3f2fa8a9-5005-4054-94ca-b68986b62750\") " pod="openshift-service-ca/service-ca-75d99c484-2z5m9"
Sep 06 10:30:35.497814 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.497664    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3f2fa8a9-5005-4054-94ca-b68986b62750-signing-cabundle\") pod \"service-ca-75d99c484-2z5m9\" (UID: \"3f2fa8a9-5005-4054-94ca-b68986b62750\") " pod="openshift-service-ca/service-ca-75d99c484-2z5m9"
Sep 06 10:30:35.497814 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.497695    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkvzw\" (UniqueName: \"kubernetes.io/projected/3f2fa8a9-5005-4054-94ca-b68986b62750-kube-api-access-jkvzw\") pod \"service-ca-75d99c484-2z5m9\" (UID: \"3f2fa8a9-5005-4054-94ca-b68986b62750\") " pod="openshift-service-ca/service-ca-75d99c484-2z5m9"
Sep 06 10:30:35.598871 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.598834    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3f2fa8a9-5005-4054-94ca-b68986b62750-signing-key\") pod \"service-ca-75d99c484-2z5m9\" (UID: \"3f2fa8a9-5005-4054-94ca-b68986b62750\") " pod="openshift-service-ca/service-ca-75d99c484-2z5m9"
Sep 06 10:30:35.599049 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.598898    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3f2fa8a9-5005-4054-94ca-b68986b62750-signing-cabundle\") pod \"service-ca-75d99c484-2z5m9\" (UID: \"3f2fa8a9-5005-4054-94ca-b68986b62750\") " pod="openshift-service-ca/service-ca-75d99c484-2z5m9"
Sep 06 10:30:35.599049 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.598932    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkvzw\" (UniqueName: \"kubernetes.io/projected/3f2fa8a9-5005-4054-94ca-b68986b62750-kube-api-access-jkvzw\") pod \"service-ca-75d99c484-2z5m9\" (UID: \"3f2fa8a9-5005-4054-94ca-b68986b62750\") " pod="openshift-service-ca/service-ca-75d99c484-2z5m9"
Sep 06 10:30:35.599696 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.599645    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3f2fa8a9-5005-4054-94ca-b68986b62750-signing-cabundle\") pod \"service-ca-75d99c484-2z5m9\" (UID: \"3f2fa8a9-5005-4054-94ca-b68986b62750\") " pod="openshift-service-ca/service-ca-75d99c484-2z5m9"
Sep 06 10:30:35.602370 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.602343    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3f2fa8a9-5005-4054-94ca-b68986b62750-signing-key\") pod \"service-ca-75d99c484-2z5m9\" (UID: \"3f2fa8a9-5005-4054-94ca-b68986b62750\") " pod="openshift-service-ca/service-ca-75d99c484-2z5m9"
Sep 06 10:30:35.607300 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.607254    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkvzw\" (UniqueName: \"kubernetes.io/projected/3f2fa8a9-5005-4054-94ca-b68986b62750-kube-api-access-jkvzw\") pod \"service-ca-75d99c484-2z5m9\" (UID: \"3f2fa8a9-5005-4054-94ca-b68986b62750\") " pod="openshift-service-ca/service-ca-75d99c484-2z5m9"
Sep 06 10:30:35.736846 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.736818    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t57dx_8a42dde3-cdd1-49a0-86f1-c67b65ca9cee/dns-node-resolver/0.log"
Sep 06 10:30:35.739065 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:35.739036    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-2z5m9"
Sep 06 10:30:36.731365 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:36.731341    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-st44k_d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3/node-ca/0.log"
Sep 06 10:30:37.131310 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:37.131280    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-74t29_0f62f367-e8fd-4f68-95ba-50376d52e32a/serve-healthcheck-canary/0.log"
Sep 06 10:30:37.315835 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:37.315801    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dace2beb-5038-4b5b-936c-120b52c07ed4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:37.315992 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:37.315973    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 10:30:37.316033 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:37.315990    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dace2beb-5038-4b5b-936c-120b52c07ed4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:37.316079 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:37.316039    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dace2beb-5038-4b5b-936c-120b52c07ed4-insights-runtime-extractor-tls podName:dace2beb-5038-4b5b-936c-120b52c07ed4 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:41.316019201 +0000 UTC m=+53.765799963 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dace2beb-5038-4b5b-936c-120b52c07ed4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pfxj6" (UID: "dace2beb-5038-4b5b-936c-120b52c07ed4") : secret "insights-runtime-extractor-tls" not found
Sep 06 10:30:37.316137 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:37.316076    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 10:30:37.316137 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:37.316128    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dace2beb-5038-4b5b-936c-120b52c07ed4-kube-rbac-proxy-cm podName:dace2beb-5038-4b5b-936c-120b52c07ed4 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:41.316112104 +0000 UTC m=+53.765892854 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dace2beb-5038-4b5b-936c-120b52c07ed4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pfxj6" (UID: "dace2beb-5038-4b5b-936c-120b52c07ed4") : configmap "kube-rbac-proxy" not found
Sep 06 10:30:37.732391 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:37.732356    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-7jr4k_99fa6d2e-adc6-4bd8-b6b0-e73045c36405/kube-storage-version-migrator-operator/0.log"
Sep 06 10:30:38.931260 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:38.931221    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:38.931732 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:38.931331    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-zmk6j\" (UID: \"4e55abd8-3289-4fff-8d8f-34129da7fb69\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"
Sep 06 10:30:38.931732 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:38.931421    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 06 10:30:38.931732 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:38.931446    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b78f8f8f-f8nfj: secret "image-registry-tls" not found
Sep 06 10:30:38.931732 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:38.931463    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 06 10:30:38.931732 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:38.931520    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls podName:69d7eb68-0175-46cd-9464-19e90cdcc1a3 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:54.931499661 +0000 UTC m=+67.381280421 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls") pod "image-registry-7b78f8f8f-f8nfj" (UID: "69d7eb68-0175-46cd-9464-19e90cdcc1a3") : secret "image-registry-tls" not found
Sep 06 10:30:38.931732 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:38.931540    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls podName:4e55abd8-3289-4fff-8d8f-34129da7fb69 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:54.93152997 +0000 UTC m=+67.381310722 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-zmk6j" (UID: "4e55abd8-3289-4fff-8d8f-34129da7fb69") : secret "cluster-monitoring-operator-tls" not found
Sep 06 10:30:39.032367 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:39.032323    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls\") pod \"dns-default-dltkp\" (UID: \"ec03af21-0135-4ba1-b4be-06adf19748ed\") " pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:39.032585 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:39.032395    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:39.032585 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:39.032447    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:39.032585 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:39.032512    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 10:30:39.032585 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:39.032564    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 06 10:30:39.032797 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:39.032589    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls podName:ec03af21-0135-4ba1-b4be-06adf19748ed nodeName:}" failed. No retries permitted until 2026-09-06 10:30:55.032567866 +0000 UTC m=+67.482348627 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls") pod "dns-default-dltkp" (UID: "ec03af21-0135-4ba1-b4be-06adf19748ed") : secret "dns-default-metrics-tls" not found
Sep 06 10:30:39.032797 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:39.032615    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-7jsrm\" (UID: \"c2cbcefd-af14-4314-a52c-674e6775c298\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm"
Sep 06 10:30:39.032797 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:39.032655    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle podName:0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c nodeName:}" failed. No retries permitted until 2026-09-06 10:30:55.032638959 +0000 UTC m=+67.482419730 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle") pod "router-default-78f4854998-kj5tp" (UID: "0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c") : configmap references non-existent config key: service-ca.crt
Sep 06 10:30:39.032797 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:39.032688    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 06 10:30:39.032797 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:39.032691    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs podName:0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c nodeName:}" failed. No retries permitted until 2026-09-06 10:30:55.032681096 +0000 UTC m=+67.482461842 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs") pod "router-default-78f4854998-kj5tp" (UID: "0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c") : secret "router-metrics-certs-default" not found
Sep 06 10:30:39.032797 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:39.032758    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls podName:c2cbcefd-af14-4314-a52c-674e6775c298 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:55.032742907 +0000 UTC m=+67.482523659 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-7jsrm" (UID: "c2cbcefd-af14-4314-a52c-674e6775c298") : secret "samples-operator-tls" not found
Sep 06 10:30:40.379353 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:40.379324    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-k9kzz" event={"ID":"840d1c17-0109-4541-93cb-7409823e8f70","Type":"ContainerStarted","Data":"f57c028ae0ffceae8a63dcad51733d02e9e415e2337eeca45eb2943e6cf417ec"}
Sep 06 10:30:40.379720 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:40.379386    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-k9kzz" event={"ID":"840d1c17-0109-4541-93cb-7409823e8f70","Type":"ContainerStarted","Data":"2a5edc6bd93764e846776abbb0430bdc0da295e18caa289406483818c1b2acb9"}
Sep 06 10:30:40.380766 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:40.380745    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl" event={"ID":"17a46f13-f538-4f23-90b8-654975a915d8","Type":"ContainerStarted","Data":"9ee83c8844a274dbef1304f1ee5c8a90187fac89b9504b7745a4794a84b82245"}
Sep 06 10:30:40.382618 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:40.382594    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl" event={"ID":"7c24bfc5-c910-4ef3-9d93-d27221a83328","Type":"ContainerStarted","Data":"c349baa0b291a5b73d5f0c470415ee825ddf2e012a42aaefcf8ba7a8646504d9"}
Sep 06 10:30:40.382695 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:40.382636    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl" event={"ID":"7c24bfc5-c910-4ef3-9d93-d27221a83328","Type":"ContainerStarted","Data":"b586d9da40a39c69be198d4f80d19626c7e9105d9c9f5d47d510720a023029d6"}
Sep 06 10:30:40.383996 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:40.383942    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bf9d9b777-fwthv" event={"ID":"eb2d3f1b-bdba-4c1d-a75f-9dbefc69f9ec","Type":"ContainerStarted","Data":"cbb254e896dc6796c8715678cbc463fa4b9836a93c9f948097c4573f02a40640"}
Sep 06 10:30:40.384341 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:40.384301    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bf9d9b777-fwthv"
Sep 06 10:30:40.385711 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:40.385690    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bf9d9b777-fwthv"
Sep 06 10:30:40.408103 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:40.407954    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-k9kzz" podStartSLOduration=0.875048058 podStartE2EDuration="7.407942697s" totalImagesPullingTime="6.532894639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:40.407621819 +0000 UTC m=+52.857402588" watchObservedRunningTime="2026-09-06 10:30:40.407942697 +0000 UTC m=+52.857723526"
Sep 06 10:30:40.433591 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:40.433528    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bf9d9b777-fwthv" podStartSLOduration=1.06644376 podStartE2EDuration="8.43347762s" totalImagesPullingTime="7.36703386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:40.431468404 +0000 UTC m=+52.881249171" watchObservedRunningTime="2026-09-06 10:30:40.43347762 +0000 UTC m=+52.883258389"
Sep 06 10:30:40.452652 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:40.452580    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9dc489cd-kfwpl" podStartSLOduration=1.05654989 podStartE2EDuration="8.452566565s" totalImagesPullingTime="7.396016675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:40.45177421 +0000 UTC m=+52.901554980" watchObservedRunningTime="2026-09-06 10:30:40.452566565 +0000 UTC m=+52.902347333"
Sep 06 10:30:40.473472 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:40.473446    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-2z5m9"]
Sep 06 10:30:40.474573 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:40.474554    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f2fa8a9_5005_4054_94ca_b68986b62750.slice/crio-5597043a072c1fc516c09f5f0cf0c7f320fe2455e830f58765c0c73ec0fc8178 WatchSource:0}: Error finding container 5597043a072c1fc516c09f5f0cf0c7f320fe2455e830f58765c0c73ec0fc8178: Status 404 returned error can't find the container with id 5597043a072c1fc516c09f5f0cf0c7f320fe2455e830f58765c0c73ec0fc8178
Sep 06 10:30:40.954106 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:40.954073    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qh488\" (UID: \"2b5a6c6a-2f52-4d65-8000-c39f44942495\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qh488"
Sep 06 10:30:40.954257 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:40.954230    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 06 10:30:40.954304 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:40.954291    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert podName:2b5a6c6a-2f52-4d65-8000-c39f44942495 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:56.954278271 +0000 UTC m=+69.404059017 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-qh488" (UID: "2b5a6c6a-2f52-4d65-8000-c39f44942495") : secret "networking-console-plugin-cert" not found
Sep 06 10:30:41.357740 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:41.357710    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dace2beb-5038-4b5b-936c-120b52c07ed4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:41.357899 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:41.357825    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dace2beb-5038-4b5b-936c-120b52c07ed4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:41.357997 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:41.357975    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 10:30:41.358068 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:41.358056    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dace2beb-5038-4b5b-936c-120b52c07ed4-insights-runtime-extractor-tls podName:dace2beb-5038-4b5b-936c-120b52c07ed4 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:49.35803743 +0000 UTC m=+61.807818196 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dace2beb-5038-4b5b-936c-120b52c07ed4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pfxj6" (UID: "dace2beb-5038-4b5b-936c-120b52c07ed4") : secret "insights-runtime-extractor-tls" not found
Sep 06 10:30:41.358292 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:41.358273    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dace2beb-5038-4b5b-936c-120b52c07ed4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:41.388663 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:41.388636    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-2z5m9" event={"ID":"3f2fa8a9-5005-4054-94ca-b68986b62750","Type":"ContainerStarted","Data":"98deb6258e1a325006259b6fc4aa960fb529168c1e6fa20f2aafd42740a31ffd"}
Sep 06 10:30:41.389054 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:41.388690    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-2z5m9" event={"ID":"3f2fa8a9-5005-4054-94ca-b68986b62750","Type":"ContainerStarted","Data":"5597043a072c1fc516c09f5f0cf0c7f320fe2455e830f58765c0c73ec0fc8178"}
Sep 06 10:30:41.390858 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:41.390825    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl" event={"ID":"7c24bfc5-c910-4ef3-9d93-d27221a83328","Type":"ContainerStarted","Data":"126ca7976e3a412f4d1880d6242c27abca85934c1cb1b1c5e8c5dd61df1e1561"}
Sep 06 10:30:41.408441 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:41.408400    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-2z5m9" podStartSLOduration=6.40838798 podStartE2EDuration="6.40838798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:41.407557036 +0000 UTC m=+53.857337806" watchObservedRunningTime="2026-09-06 10:30:41.40838798 +0000 UTC m=+53.858168747"
Sep 06 10:30:41.427080 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:41.427039    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl" podStartSLOduration=2.083936531 podStartE2EDuration="9.427025615s" totalImagesPullingTime="7.343089084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:41.425774975 +0000 UTC m=+53.875555744" watchObservedRunningTime="2026-09-06 10:30:41.427025615 +0000 UTC m=+53.876806383"
Sep 06 10:30:42.525023 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:42.524960    2557 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl" podUID="7c24bfc5-c910-4ef3-9d93-d27221a83328" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 06 10:30:47.257425 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:47.257397    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tw474"
Sep 06 10:30:49.430244 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:49.430214    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dace2beb-5038-4b5b-936c-120b52c07ed4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:49.432393 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:49.432362    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dace2beb-5038-4b5b-936c-120b52c07ed4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pfxj6\" (UID: \"dace2beb-5038-4b5b-936c-120b52c07ed4\") " pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:49.501149 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:49.501118    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-l4ftj\""
Sep 06 10:30:49.508292 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:49.508271    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pfxj6"
Sep 06 10:30:49.776716 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:49.776691    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pfxj6"]
Sep 06 10:30:49.778899 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:49.778873    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddace2beb_5038_4b5b_936c_120b52c07ed4.slice/crio-c11115b5c4409f11b73841e1a529e9a3617ef05fcb25561c1fc7af3f8c320c27 WatchSource:0}: Error finding container c11115b5c4409f11b73841e1a529e9a3617ef05fcb25561c1fc7af3f8c320c27: Status 404 returned error can't find the container with id c11115b5c4409f11b73841e1a529e9a3617ef05fcb25561c1fc7af3f8c320c27
Sep 06 10:30:50.031038 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:50.030973    2557 scope.go:117] "RemoveContainer" containerID="f48248402720e700b261aaf78a26f909b7529dce6e6a3408289dd6613214d744"
Sep 06 10:30:50.424736 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:50.424697    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pfxj6" event={"ID":"dace2beb-5038-4b5b-936c-120b52c07ed4","Type":"ContainerStarted","Data":"3db3062796abde280addaa271cce6772fcfa4d42d95b81a12ef3c14427729039"}
Sep 06 10:30:50.424900 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:50.424769    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pfxj6" event={"ID":"dace2beb-5038-4b5b-936c-120b52c07ed4","Type":"ContainerStarted","Data":"c11115b5c4409f11b73841e1a529e9a3617ef05fcb25561c1fc7af3f8c320c27"}
Sep 06 10:30:50.426270 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:50.426250    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/2.log"
Sep 06 10:30:50.426657 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:50.426644    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/1.log"
Sep 06 10:30:50.426706 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:50.426672    2557 generic.go:356] "Generic (PLEG): container finished" podID="938e293f-7414-4bdd-946f-0b463ddc8ecb" containerID="4266d852c1ddfdf8cdc5743b9c733ab5a54212e5315a5b36c2910d4df1ee022a" exitCode=255
Sep 06 10:30:50.426706 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:50.426697    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-9lt24" event={"ID":"938e293f-7414-4bdd-946f-0b463ddc8ecb","Type":"ContainerDied","Data":"4266d852c1ddfdf8cdc5743b9c733ab5a54212e5315a5b36c2910d4df1ee022a"}
Sep 06 10:30:50.426803 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:50.426731    2557 scope.go:117] "RemoveContainer" containerID="f48248402720e700b261aaf78a26f909b7529dce6e6a3408289dd6613214d744"
Sep 06 10:30:50.427057 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:50.427041    2557 scope.go:117] "RemoveContainer" containerID="4266d852c1ddfdf8cdc5743b9c733ab5a54212e5315a5b36c2910d4df1ee022a"
Sep 06 10:30:50.427253 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:50.427224    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-9lt24_openshift-console-operator(938e293f-7414-4bdd-946f-0b463ddc8ecb)\"" pod="openshift-console-operator/console-operator-85b454874b-9lt24" podUID="938e293f-7414-4bdd-946f-0b463ddc8ecb"
Sep 06 10:30:51.431712 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:51.431683    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/2.log"
Sep 06 10:30:51.433613 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:51.433585    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pfxj6" event={"ID":"dace2beb-5038-4b5b-936c-120b52c07ed4","Type":"ContainerStarted","Data":"89a07d71d1aa465f75113b2361958594eca13db9a19747fac44ad057798a999d"}
Sep 06 10:30:52.523439 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:52.523400    2557 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl" podUID="7c24bfc5-c910-4ef3-9d93-d27221a83328" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 06 10:30:53.405606 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:53.405574    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:30:53.405774 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:53.405630    2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:30:53.405925 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:53.405910    2557 scope.go:117] "RemoveContainer" containerID="4266d852c1ddfdf8cdc5743b9c733ab5a54212e5315a5b36c2910d4df1ee022a"
Sep 06 10:30:53.406071 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:30:53.406054    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-9lt24_openshift-console-operator(938e293f-7414-4bdd-946f-0b463ddc8ecb)\"" pod="openshift-console-operator/console-operator-85b454874b-9lt24" podUID="938e293f-7414-4bdd-946f-0b463ddc8ecb"
Sep 06 10:30:53.446797 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:53.446768    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pfxj6" event={"ID":"dace2beb-5038-4b5b-936c-120b52c07ed4","Type":"ContainerStarted","Data":"6ad0086e27bc6a8d3ee0e1154869a6931b335c432f373cf19ec604f95101d9dd"}
Sep 06 10:30:53.462969 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:53.462924    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pfxj6" podStartSLOduration=17.688309976 podStartE2EDuration="20.462910436s" totalImagesPullingTime="2.77460046s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:30:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:53.462357574 +0000 UTC m=+65.912138341" watchObservedRunningTime="2026-09-06 10:30:53.462910436 +0000 UTC m=+65.912691208"
Sep 06 10:30:53.873837 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:53.873807    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs\") pod \"network-metrics-daemon-pgxvx\" (UID: \"5ef2cd14-0fae-417c-8d8a-b407e0c03903\") " pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:30:53.874199 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:53.873872    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret\") pod \"global-pull-secret-syncer-hbgj5\" (UID: \"503a2587-73cc-43a0-b058-242b6b7ec81b\") " pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:30:53.876407 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:53.876390    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 06 10:30:53.876484 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:53.876469    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 06 10:30:53.886622 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:53.886603    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/503a2587-73cc-43a0-b058-242b6b7ec81b-original-pull-secret\") pod \"global-pull-secret-syncer-hbgj5\" (UID: \"503a2587-73cc-43a0-b058-242b6b7ec81b\") " pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:30:53.886704 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:53.886619    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ef2cd14-0fae-417c-8d8a-b407e0c03903-metrics-certs\") pod \"network-metrics-daemon-pgxvx\" (UID: \"5ef2cd14-0fae-417c-8d8a-b407e0c03903\") " pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:30:53.972302 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:53.972276    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hbgj5"
Sep 06 10:30:53.974771 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:53.974412    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngkp5\" (UniqueName: \"kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5\") pod \"network-check-target-bcjj6\" (UID: \"96763caf-341a-40a7-ba78-c94b6dedebfd\") " pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:30:53.976657 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:53.976641    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngkp5\" (UniqueName: \"kubernetes.io/projected/96763caf-341a-40a7-ba78-c94b6dedebfd-kube-api-access-ngkp5\") pod \"network-check-target-bcjj6\" (UID: \"96763caf-341a-40a7-ba78-c94b6dedebfd\") " pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:30:53.980036 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:53.980016    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h4kvp\""
Sep 06 10:30:53.988363 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:53.988342    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:30:53.991965 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:53.991944    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bx9l2\""
Sep 06 10:30:53.999887 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:53.999867    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgxvx"
Sep 06 10:30:54.245454 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:54.245429    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hbgj5"]
Sep 06 10:30:54.247687 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:54.247663    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod503a2587_73cc_43a0_b058_242b6b7ec81b.slice/crio-bb87bc39287de92ccb6c07da41c854ebeb0fe07c967522d2be773a03e858722a WatchSource:0}: Error finding container bb87bc39287de92ccb6c07da41c854ebeb0fe07c967522d2be773a03e858722a: Status 404 returned error can't find the container with id bb87bc39287de92ccb6c07da41c854ebeb0fe07c967522d2be773a03e858722a
Sep 06 10:30:54.268854 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:54.268832    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bcjj6"]
Sep 06 10:30:54.271384 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:54.271361    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96763caf_341a_40a7_ba78_c94b6dedebfd.slice/crio-173189758f8da51ad7386827552e167ffe1c13687a377d7f3fd6e2d47b627a9d WatchSource:0}: Error finding container 173189758f8da51ad7386827552e167ffe1c13687a377d7f3fd6e2d47b627a9d: Status 404 returned error can't find the container with id 173189758f8da51ad7386827552e167ffe1c13687a377d7f3fd6e2d47b627a9d
Sep 06 10:30:54.280306 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:54.280284    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pgxvx"]
Sep 06 10:30:54.282899 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:54.282878    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ef2cd14_0fae_417c_8d8a_b407e0c03903.slice/crio-37bdee6abc5e4539e00142344f16ebaeff9c54f31e0cb0711372ad70ff3bab6a WatchSource:0}: Error finding container 37bdee6abc5e4539e00142344f16ebaeff9c54f31e0cb0711372ad70ff3bab6a: Status 404 returned error can't find the container with id 37bdee6abc5e4539e00142344f16ebaeff9c54f31e0cb0711372ad70ff3bab6a
Sep 06 10:30:54.451136 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:54.451109    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pgxvx" event={"ID":"5ef2cd14-0fae-417c-8d8a-b407e0c03903","Type":"ContainerStarted","Data":"37bdee6abc5e4539e00142344f16ebaeff9c54f31e0cb0711372ad70ff3bab6a"}
Sep 06 10:30:54.452594 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:54.452571    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bcjj6" event={"ID":"96763caf-341a-40a7-ba78-c94b6dedebfd","Type":"ContainerStarted","Data":"b9cda27bbff2049dbeee660510a907df64703d691ec8e1582e4f207a8859149b"}
Sep 06 10:30:54.452743 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:54.452718    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:30:54.452837 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:54.452761    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bcjj6" event={"ID":"96763caf-341a-40a7-ba78-c94b6dedebfd","Type":"ContainerStarted","Data":"173189758f8da51ad7386827552e167ffe1c13687a377d7f3fd6e2d47b627a9d"}
Sep 06 10:30:54.453812 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:54.453789    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hbgj5" event={"ID":"503a2587-73cc-43a0-b058-242b6b7ec81b","Type":"ContainerStarted","Data":"bb87bc39287de92ccb6c07da41c854ebeb0fe07c967522d2be773a03e858722a"}
Sep 06 10:30:54.467348 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:54.467309    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bcjj6" podStartSLOduration=66.467296699 podStartE2EDuration="1m6.467296699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:29:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:54.46624113 +0000 UTC m=+66.916021900" watchObservedRunningTime="2026-09-06 10:30:54.467296699 +0000 UTC m=+66.917077491"
Sep 06 10:30:54.983104 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:54.983063    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-zmk6j\" (UID: \"4e55abd8-3289-4fff-8d8f-34129da7fb69\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"
Sep 06 10:30:54.983561 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:54.983204    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:54.985994 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:54.985965    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e55abd8-3289-4fff-8d8f-34129da7fb69-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-zmk6j\" (UID: \"4e55abd8-3289-4fff-8d8f-34129da7fb69\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"
Sep 06 10:30:54.986110 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:54.986043    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls\") pod \"image-registry-7b78f8f8f-f8nfj\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") " pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:55.084336 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.084293    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-7jsrm\" (UID: \"c2cbcefd-af14-4314-a52c-674e6775c298\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm"
Sep 06 10:30:55.084482 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.084371    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls\") pod \"dns-default-dltkp\" (UID: \"ec03af21-0135-4ba1-b4be-06adf19748ed\") " pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:55.084482 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.084401    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:55.084482 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.084445    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:55.085182 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.085152    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-service-ca-bundle\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:55.087789 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.087764    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2cbcefd-af14-4314-a52c-674e6775c298-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-7jsrm\" (UID: \"c2cbcefd-af14-4314-a52c-674e6775c298\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm"
Sep 06 10:30:55.087881 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.087774    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c-metrics-certs\") pod \"router-default-78f4854998-kj5tp\" (UID: \"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c\") " pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:55.088187 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.088147    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec03af21-0135-4ba1-b4be-06adf19748ed-metrics-tls\") pod \"dns-default-dltkp\" (UID: \"ec03af21-0135-4ba1-b4be-06adf19748ed\") " pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:55.101618 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.101597    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ld7ld\""
Sep 06 10:30:55.109569 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.109545    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"
Sep 06 10:30:55.148394 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.148369    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f2fgs\""
Sep 06 10:30:55.156237 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.156216    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:55.185605 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.185581    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tdqz7\""
Sep 06 10:30:55.193261 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.193243    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm"
Sep 06 10:30:55.236484 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.236418    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-89vv4\""
Sep 06 10:30:55.244942 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.244910    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:55.247166 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.247142    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-89vqh\""
Sep 06 10:30:55.256188 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.256094    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dltkp"
Sep 06 10:30:55.659276 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.659208    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-78f4854998-kj5tp"]
Sep 06 10:30:55.664789 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:55.663837    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e89829b_4d1d_4a6a_b4d1_ef2601dbcd0c.slice/crio-c108b63d8ec297ed087b1a7e807651ffdfec6ee3789804fadb65da3118e543d0 WatchSource:0}: Error finding container c108b63d8ec297ed087b1a7e807651ffdfec6ee3789804fadb65da3118e543d0: Status 404 returned error can't find the container with id c108b63d8ec297ed087b1a7e807651ffdfec6ee3789804fadb65da3118e543d0
Sep 06 10:30:55.665955 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.665926    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b78f8f8f-f8nfj"]
Sep 06 10:30:55.668660 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:55.668634    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69d7eb68_0175_46cd_9464_19e90cdcc1a3.slice/crio-b9b0b48285cf7e035559f085bd3399c83d10cea8c6d311573578f6b4b0be94d4 WatchSource:0}: Error finding container b9b0b48285cf7e035559f085bd3399c83d10cea8c6d311573578f6b4b0be94d4: Status 404 returned error can't find the container with id b9b0b48285cf7e035559f085bd3399c83d10cea8c6d311573578f6b4b0be94d4
Sep 06 10:30:55.677593 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.677531    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm"]
Sep 06 10:30:55.680083 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.680064    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j"]
Sep 06 10:30:55.684751 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:55.684695    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e55abd8_3289_4fff_8d8f_34129da7fb69.slice/crio-fa2a77b40f478bb98b701102e1a22a7a64038f8a77ccd1480693babd289ff111 WatchSource:0}: Error finding container fa2a77b40f478bb98b701102e1a22a7a64038f8a77ccd1480693babd289ff111: Status 404 returned error can't find the container with id fa2a77b40f478bb98b701102e1a22a7a64038f8a77ccd1480693babd289ff111
Sep 06 10:30:55.707852 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:55.707755    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dltkp"]
Sep 06 10:30:55.710862 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:30:55.710835    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec03af21_0135_4ba1_b4be_06adf19748ed.slice/crio-9ec74ad87304d8e4a699df557ee5c5cf621274c577ae334dc4de12ac38fc5b8f WatchSource:0}: Error finding container 9ec74ad87304d8e4a699df557ee5c5cf621274c577ae334dc4de12ac38fc5b8f: Status 404 returned error can't find the container with id 9ec74ad87304d8e4a699df557ee5c5cf621274c577ae334dc4de12ac38fc5b8f
Sep 06 10:30:56.470850 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:56.470770    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pgxvx" event={"ID":"5ef2cd14-0fae-417c-8d8a-b407e0c03903","Type":"ContainerStarted","Data":"0ed29874557d41c797aacd3e354e931f57b6dcb5f387785ba9ec04c0b430faad"}
Sep 06 10:30:56.470850 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:56.470836    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pgxvx" event={"ID":"5ef2cd14-0fae-417c-8d8a-b407e0c03903","Type":"ContainerStarted","Data":"38c18ca7115345f8964a519010bca57262b07571789abdb2334010e0b971e306"}
Sep 06 10:30:56.473368 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:56.473294    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dltkp" event={"ID":"ec03af21-0135-4ba1-b4be-06adf19748ed","Type":"ContainerStarted","Data":"9ec74ad87304d8e4a699df557ee5c5cf621274c577ae334dc4de12ac38fc5b8f"}
Sep 06 10:30:56.476354 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:56.475741    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj" event={"ID":"69d7eb68-0175-46cd-9464-19e90cdcc1a3","Type":"ContainerStarted","Data":"9a6fa712d82dcd82bd882b92a7e5d874f890316170d62afb8f5098b82bb57805"}
Sep 06 10:30:56.476354 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:56.475784    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj" event={"ID":"69d7eb68-0175-46cd-9464-19e90cdcc1a3","Type":"ContainerStarted","Data":"b9b0b48285cf7e035559f085bd3399c83d10cea8c6d311573578f6b4b0be94d4"}
Sep 06 10:30:56.476354 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:56.476295    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:30:56.478206 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:56.478126    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm" event={"ID":"c2cbcefd-af14-4314-a52c-674e6775c298","Type":"ContainerStarted","Data":"b2b61f73253070a676a17a234ab091403e41e48c964d9eaf4ce054403fffea04"}
Sep 06 10:30:56.480009 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:56.479952    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j" event={"ID":"4e55abd8-3289-4fff-8d8f-34129da7fb69","Type":"ContainerStarted","Data":"fa2a77b40f478bb98b701102e1a22a7a64038f8a77ccd1480693babd289ff111"}
Sep 06 10:30:56.482256 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:56.482164    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-78f4854998-kj5tp" event={"ID":"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c","Type":"ContainerStarted","Data":"d308ffbd50e7ee471e294367613744e22eb8d58cd8d2ece20215920b45fb4b1d"}
Sep 06 10:30:56.482256 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:56.482210    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-78f4854998-kj5tp" event={"ID":"0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c","Type":"ContainerStarted","Data":"c108b63d8ec297ed087b1a7e807651ffdfec6ee3789804fadb65da3118e543d0"}
Sep 06 10:30:56.487325 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:56.487278    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pgxvx" podStartSLOduration=67.390428505 podStartE2EDuration="1m8.487259785s" totalImagesPullingTime="1.09683128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:29:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:56.485389643 +0000 UTC m=+68.935170414" watchObservedRunningTime="2026-09-06 10:30:56.487259785 +0000 UTC m=+68.937040554"
Sep 06 10:30:56.527455 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:56.527394    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-78f4854998-kj5tp" podStartSLOduration=37.527375256 podStartE2EDuration="37.527375256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:56.525015699 +0000 UTC m=+68.974796468" watchObservedRunningTime="2026-09-06 10:30:56.527375256 +0000 UTC m=+68.977156025"
Sep 06 10:30:56.527657 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:56.527626    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj" podStartSLOduration=68.527618209 podStartE2EDuration="1m8.527618209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:29:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:56.50804237 +0000 UTC m=+68.957823140" watchObservedRunningTime="2026-09-06 10:30:56.527618209 +0000 UTC m=+68.977398958"
Sep 06 10:30:57.002986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:57.002254    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qh488\" (UID: \"2b5a6c6a-2f52-4d65-8000-c39f44942495\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qh488"
Sep 06 10:30:57.006221 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:57.006125    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2b5a6c6a-2f52-4d65-8000-c39f44942495-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qh488\" (UID: \"2b5a6c6a-2f52-4d65-8000-c39f44942495\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qh488"
Sep 06 10:30:57.224122 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:57.224080    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-lrlxd\""
Sep 06 10:30:57.231305 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:57.231275    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-qh488"
Sep 06 10:30:57.245244 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:57.245221    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:57.248171 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:57.248147    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:57.487225 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:57.487190    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:30:57.488116 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:30:57.488095    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-78f4854998-kj5tp"
Sep 06 10:31:00.514312 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:00.512918    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dltkp" event={"ID":"ec03af21-0135-4ba1-b4be-06adf19748ed","Type":"ContainerStarted","Data":"c9ea8bbb65f701fa2dda43dcc8be82033b69cd5deb2b2b95edd5f0348fa07e56"}
Sep 06 10:31:00.516876 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:00.516804    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm" event={"ID":"c2cbcefd-af14-4314-a52c-674e6775c298","Type":"ContainerStarted","Data":"42853af2eb18fbb9d32a99cc544cd63afbd7f60db0f55f91b9fc707e502ce2f5"}
Sep 06 10:31:00.521096 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:00.520386    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j" event={"ID":"4e55abd8-3289-4fff-8d8f-34129da7fb69","Type":"ContainerStarted","Data":"629ec084d4d49ffe767f1bf1d87bdb6d15e99f38aaaf2a1a4078031b0762d579"}
Sep 06 10:31:00.524748 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:00.524026    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hbgj5" event={"ID":"503a2587-73cc-43a0-b058-242b6b7ec81b","Type":"ContainerStarted","Data":"4a106e7d32819187749be8dd0cca205620e3c534a7b22a88201bf24808c82e91"}
Sep 06 10:31:00.539219 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:00.539170    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zmk6j" podStartSLOduration=36.955276379 podStartE2EDuration="41.539152371s" totalImagesPullingTime="4.583875992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:00.537980945 +0000 UTC m=+72.987761714" watchObservedRunningTime="2026-09-06 10:31:00.539152371 +0000 UTC m=+72.988933140"
Sep 06 10:31:00.553208 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:00.552902    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-qh488"]
Sep 06 10:31:00.557959 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:31:00.557931    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b5a6c6a_2f52_4d65_8000_c39f44942495.slice/crio-1bf7f64208d4dbeddf27dce75592306e9860292d46c46f379ff93820f0ef8f78 WatchSource:0}: Error finding container 1bf7f64208d4dbeddf27dce75592306e9860292d46c46f379ff93820f0ef8f78: Status 404 returned error can't find the container with id 1bf7f64208d4dbeddf27dce75592306e9860292d46c46f379ff93820f0ef8f78
Sep 06 10:31:01.533389 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:01.533273    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm" event={"ID":"c2cbcefd-af14-4314-a52c-674e6775c298","Type":"ContainerStarted","Data":"9758f6993162e4c22846b3a3403d0962049e8356fbdc39306b27f26e082cd03e"}
Sep 06 10:31:01.534755 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:01.534721    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-qh488" event={"ID":"2b5a6c6a-2f52-4d65-8000-c39f44942495","Type":"ContainerStarted","Data":"1bf7f64208d4dbeddf27dce75592306e9860292d46c46f379ff93820f0ef8f78"}
Sep 06 10:31:01.536758 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:01.536720    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dltkp" event={"ID":"ec03af21-0135-4ba1-b4be-06adf19748ed","Type":"ContainerStarted","Data":"9f4731b933cea0ce9e8eda250eb3d48290d2903cd7696496eb8747981abb213d"}
Sep 06 10:31:01.549680 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:01.549618    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hbgj5" podStartSLOduration=67.513752523 podStartE2EDuration="1m13.549602588s" totalImagesPullingTime="6.035850065s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:29:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:00.555682029 +0000 UTC m=+73.005462808" watchObservedRunningTime="2026-09-06 10:31:01.549602588 +0000 UTC m=+73.999383357"
Sep 06 10:31:01.568815 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:01.568759    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7jsrm" podStartSLOduration=38.028850567 podStartE2EDuration="42.568742075s" totalImagesPullingTime="4.539891508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:01.550307232 +0000 UTC m=+74.000088003" watchObservedRunningTime="2026-09-06 10:31:01.568742075 +0000 UTC m=+74.018522843"
Sep 06 10:31:01.569358 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:01.569319    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dltkp" podStartSLOduration=35.009580713 podStartE2EDuration="39.56931073s" totalImagesPullingTime="4.559730017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:01.568005007 +0000 UTC m=+74.017785776" watchObservedRunningTime="2026-09-06 10:31:01.56931073 +0000 UTC m=+74.019091500"
Sep 06 10:31:02.524688 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:02.524614    2557 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl" podUID="7c24bfc5-c910-4ef3-9d93-d27221a83328" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 06 10:31:02.524688 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:02.524685    2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl"
Sep 06 10:31:02.525191 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:02.525156    2557 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"126ca7976e3a412f4d1880d6242c27abca85934c1cb1b1c5e8c5dd61df1e1561"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 06 10:31:02.525289 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:02.525212    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl" podUID="7c24bfc5-c910-4ef3-9d93-d27221a83328" containerName="service-proxy" containerID="cri-o://126ca7976e3a412f4d1880d6242c27abca85934c1cb1b1c5e8c5dd61df1e1561" gracePeriod=30
Sep 06 10:31:02.541798 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:02.541767    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-qh488" event={"ID":"2b5a6c6a-2f52-4d65-8000-c39f44942495","Type":"ContainerStarted","Data":"0900f818c2d8092c9feeb5f29aea9aa0533403b38048e75721f09cb679c8bbc5"}
Sep 06 10:31:02.542134 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:02.541987    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dltkp"
Sep 06 10:31:02.555747 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:02.555696    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-qh488" podStartSLOduration=35.900328468 podStartE2EDuration="37.555681558s" totalImagesPullingTime="1.65535309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:02.555519211 +0000 UTC m=+75.005300007" watchObservedRunningTime="2026-09-06 10:31:02.555681558 +0000 UTC m=+75.005462326"
Sep 06 10:31:03.548740 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:03.548704    2557 generic.go:356] "Generic (PLEG): container finished" podID="7c24bfc5-c910-4ef3-9d93-d27221a83328" containerID="126ca7976e3a412f4d1880d6242c27abca85934c1cb1b1c5e8c5dd61df1e1561" exitCode=2
Sep 06 10:31:03.549095 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:03.548775    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl" event={"ID":"7c24bfc5-c910-4ef3-9d93-d27221a83328","Type":"ContainerDied","Data":"126ca7976e3a412f4d1880d6242c27abca85934c1cb1b1c5e8c5dd61df1e1561"}
Sep 06 10:31:03.549095 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:03.548829    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c95667689-qnzwl" event={"ID":"7c24bfc5-c910-4ef3-9d93-d27221a83328","Type":"ContainerStarted","Data":"16fd9c3098304cdbeb73a6f58fa1400ad9597942baa8e015390ff30ef980f945"}
Sep 06 10:31:04.030712 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:04.030684    2557 scope.go:117] "RemoveContainer" containerID="4266d852c1ddfdf8cdc5743b9c733ab5a54212e5315a5b36c2910d4df1ee022a"
Sep 06 10:31:04.030870 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:31:04.030841    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-9lt24_openshift-console-operator(938e293f-7414-4bdd-946f-0b463ddc8ecb)\"" pod="openshift-console-operator/console-operator-85b454874b-9lt24" podUID="938e293f-7414-4bdd-946f-0b463ddc8ecb"
Sep 06 10:31:10.370183 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.370145    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-286w4"]
Sep 06 10:31:10.387427 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.387395    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.390593 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.390416    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 06 10:31:10.390593 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.390486    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 06 10:31:10.391521 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.391399    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-v6mwz\""
Sep 06 10:31:10.391521 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.391432    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 06 10:31:10.391521 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.391461    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 06 10:31:10.408729 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.408709    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/128c863a-c6d4-483e-ac5a-dc38cb426de3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.408836 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.408748    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/128c863a-c6d4-483e-ac5a-dc38cb426de3-node-exporter-accelerators-collector-config\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.408836 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.408768    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/128c863a-c6d4-483e-ac5a-dc38cb426de3-root\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.408936 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.408836    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/128c863a-c6d4-483e-ac5a-dc38cb426de3-node-exporter-wtmp\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.408936 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.408878    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/128c863a-c6d4-483e-ac5a-dc38cb426de3-sys\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.409049 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.408937    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/128c863a-c6d4-483e-ac5a-dc38cb426de3-node-exporter-textfile\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.409049 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.408957    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/128c863a-c6d4-483e-ac5a-dc38cb426de3-metrics-client-ca\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.409049 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.408977    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/128c863a-c6d4-483e-ac5a-dc38cb426de3-node-exporter-tls\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.409049 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.409034    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw8df\" (UniqueName: \"kubernetes.io/projected/128c863a-c6d4-483e-ac5a-dc38cb426de3-kube-api-access-gw8df\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.511191 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.510443    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/128c863a-c6d4-483e-ac5a-dc38cb426de3-sys\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.511191 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.510518    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/128c863a-c6d4-483e-ac5a-dc38cb426de3-sys\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.511191 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.510565    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/128c863a-c6d4-483e-ac5a-dc38cb426de3-node-exporter-textfile\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.511191 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.510603    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/128c863a-c6d4-483e-ac5a-dc38cb426de3-metrics-client-ca\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.511191 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.510634    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/128c863a-c6d4-483e-ac5a-dc38cb426de3-node-exporter-tls\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.511191 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.510679    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw8df\" (UniqueName: \"kubernetes.io/projected/128c863a-c6d4-483e-ac5a-dc38cb426de3-kube-api-access-gw8df\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.511191 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.510736    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/128c863a-c6d4-483e-ac5a-dc38cb426de3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.511191 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.510794    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/128c863a-c6d4-483e-ac5a-dc38cb426de3-node-exporter-accelerators-collector-config\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.511191 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.510829    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/128c863a-c6d4-483e-ac5a-dc38cb426de3-root\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.511778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.511253    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/128c863a-c6d4-483e-ac5a-dc38cb426de3-node-exporter-wtmp\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.511778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.511424    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/128c863a-c6d4-483e-ac5a-dc38cb426de3-node-exporter-wtmp\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.511778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.511755    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/128c863a-c6d4-483e-ac5a-dc38cb426de3-node-exporter-textfile\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.517700 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:31:10.512388    2557 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 06 10:31:10.517700 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:31:10.512468    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/128c863a-c6d4-483e-ac5a-dc38cb426de3-node-exporter-tls podName:128c863a-c6d4-483e-ac5a-dc38cb426de3 nodeName:}" failed. No retries permitted until 2026-09-06 10:31:11.012448274 +0000 UTC m=+83.462229023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/128c863a-c6d4-483e-ac5a-dc38cb426de3-node-exporter-tls") pod "node-exporter-286w4" (UID: "128c863a-c6d4-483e-ac5a-dc38cb426de3") : secret "node-exporter-tls" not found
Sep 06 10:31:10.517700 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.513315    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/128c863a-c6d4-483e-ac5a-dc38cb426de3-root\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.517700 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.513848    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/128c863a-c6d4-483e-ac5a-dc38cb426de3-node-exporter-accelerators-collector-config\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.517700 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.512290    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/128c863a-c6d4-483e-ac5a-dc38cb426de3-metrics-client-ca\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.519746 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.519714    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/128c863a-c6d4-483e-ac5a-dc38cb426de3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:10.520705 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:10.520675    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw8df\" (UniqueName: \"kubernetes.io/projected/128c863a-c6d4-483e-ac5a-dc38cb426de3-kube-api-access-gw8df\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:11.017149 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:11.017116    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/128c863a-c6d4-483e-ac5a-dc38cb426de3-node-exporter-tls\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:11.019359 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:11.019341    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/128c863a-c6d4-483e-ac5a-dc38cb426de3-node-exporter-tls\") pod \"node-exporter-286w4\" (UID: \"128c863a-c6d4-483e-ac5a-dc38cb426de3\") " pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:11.298909 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:11.298832    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-286w4"
Sep 06 10:31:11.323353 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:31:11.323323    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod128c863a_c6d4_483e_ac5a_dc38cb426de3.slice/crio-014ad39241bbc5bdd90d8bd27cad5e284bbf05087c7d8549229cec1526eebd7e WatchSource:0}: Error finding container 014ad39241bbc5bdd90d8bd27cad5e284bbf05087c7d8549229cec1526eebd7e: Status 404 returned error can't find the container with id 014ad39241bbc5bdd90d8bd27cad5e284bbf05087c7d8549229cec1526eebd7e
Sep 06 10:31:11.581391 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:11.581315    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-286w4" event={"ID":"128c863a-c6d4-483e-ac5a-dc38cb426de3","Type":"ContainerStarted","Data":"014ad39241bbc5bdd90d8bd27cad5e284bbf05087c7d8549229cec1526eebd7e"}
Sep 06 10:31:12.550865 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:12.550844    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dltkp"
Sep 06 10:31:13.593968 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:13.593936    2557 generic.go:356] "Generic (PLEG): container finished" podID="128c863a-c6d4-483e-ac5a-dc38cb426de3" containerID="ca621c85eebc54dd00e1a2dc4dafa3bcf6b27c01b2d3f695aa37f85a89c90949" exitCode=0
Sep 06 10:31:13.594367 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:13.593996    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-286w4" event={"ID":"128c863a-c6d4-483e-ac5a-dc38cb426de3","Type":"ContainerDied","Data":"ca621c85eebc54dd00e1a2dc4dafa3bcf6b27c01b2d3f695aa37f85a89c90949"}
Sep 06 10:31:14.600245 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:14.600212    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-286w4" event={"ID":"128c863a-c6d4-483e-ac5a-dc38cb426de3","Type":"ContainerStarted","Data":"ac0a9dab2bcf5680beef3ffcfbfa3be7e4ff54b3dc8cbfc3954405f51d879218"}
Sep 06 10:31:14.600665 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:14.600270    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-286w4" event={"ID":"128c863a-c6d4-483e-ac5a-dc38cb426de3","Type":"ContainerStarted","Data":"cead048f56df2c112b1f8a36dc0b69dbb9953b8c25d53e005aa4a1e4ea8ec695"}
Sep 06 10:31:14.622469 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:14.622418    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-286w4" podStartSLOduration=3.357465313 podStartE2EDuration="4.622402602s" totalImagesPullingTime="1.203402755s" totalInitContainerRuntime="61.534534ms" isStatefulPod=true podCreationTimestamp="2026-09-06 10:31:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:14.620561723 +0000 UTC m=+87.070342490" watchObservedRunningTime="2026-09-06 10:31:14.622402602 +0000 UTC m=+87.072183366"
Sep 06 10:31:15.031095 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:15.031022    2557 scope.go:117] "RemoveContainer" containerID="4266d852c1ddfdf8cdc5743b9c733ab5a54212e5315a5b36c2910d4df1ee022a"
Sep 06 10:31:15.160746 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:15.160714    2557 patch_prober.go:28] interesting pod/image-registry-7b78f8f8f-f8nfj container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 06 10:31:15.160894 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:15.160764    2557 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj" podUID="69d7eb68-0175-46cd-9464-19e90cdcc1a3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 10:31:15.605883 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:15.605857    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/2.log"
Sep 06 10:31:15.606265 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:15.605977    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-9lt24" event={"ID":"938e293f-7414-4bdd-946f-0b463ddc8ecb","Type":"ContainerStarted","Data":"83d3e64401e57b1619e8f825432356e81e6e3dce266c4d3b8cd17bcd403b4581"}
Sep 06 10:31:15.606473 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:15.606451    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:31:15.624714 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:15.624662    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-9lt24" podStartSLOduration=48.885592749 podStartE2EDuration="56.624644323s" totalImagesPullingTime="7.739051574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:15.623080911 +0000 UTC m=+88.072861680" watchObservedRunningTime="2026-09-06 10:31:15.624644323 +0000 UTC m=+88.074425094"
Sep 06 10:31:15.828442 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:15.828415    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-9lt24"
Sep 06 10:31:18.496618 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:18.496584    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:31:21.214468 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:21.214436    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b78f8f8f-f8nfj"]
Sep 06 10:31:25.466071 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:25.465996    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bcjj6"
Sep 06 10:31:42.714057 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:42.714023    2557 generic.go:356] "Generic (PLEG): container finished" podID="99fa6d2e-adc6-4bd8-b6b0-e73045c36405" containerID="c1e930b360ff0c5a1a0f732db97ae9c3f8e8011c178d22bb23fc7cc42175a87f" exitCode=0
Sep 06 10:31:42.714616 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:42.714101    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7jr4k" event={"ID":"99fa6d2e-adc6-4bd8-b6b0-e73045c36405","Type":"ContainerDied","Data":"c1e930b360ff0c5a1a0f732db97ae9c3f8e8011c178d22bb23fc7cc42175a87f"}
Sep 06 10:31:42.714616 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:42.714524    2557 scope.go:117] "RemoveContainer" containerID="c1e930b360ff0c5a1a0f732db97ae9c3f8e8011c178d22bb23fc7cc42175a87f"
Sep 06 10:31:43.720109 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:43.720071    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7jr4k" event={"ID":"99fa6d2e-adc6-4bd8-b6b0-e73045c36405","Type":"ContainerStarted","Data":"7cc0c307bfafe3296937ec2ff94934c3f99a39b1706f35f1d6bb26c0ca05cebf"}
Sep 06 10:31:46.242807 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.242758    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj" podUID="69d7eb68-0175-46cd-9464-19e90cdcc1a3" containerName="registry" containerID="cri-o://9a6fa712d82dcd82bd882b92a7e5d874f890316170d62afb8f5098b82bb57805" gracePeriod=30
Sep 06 10:31:46.494641 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.494587    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:31:46.594408 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.594381    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-bound-sa-token\") pod \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") "
Sep 06 10:31:46.594617 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.594433    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kkzl\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-kube-api-access-6kkzl\") pod \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") "
Sep 06 10:31:46.594617 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.594460    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69d7eb68-0175-46cd-9464-19e90cdcc1a3-ca-trust-extracted\") pod \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") "
Sep 06 10:31:46.594617 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.594563    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-certificates\") pod \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") "
Sep 06 10:31:46.594797 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.594629    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69d7eb68-0175-46cd-9464-19e90cdcc1a3-image-registry-private-configuration\") pod \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") "
Sep 06 10:31:46.594797 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.594659    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69d7eb68-0175-46cd-9464-19e90cdcc1a3-installation-pull-secrets\") pod \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") "
Sep 06 10:31:46.594797 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.594684    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69d7eb68-0175-46cd-9464-19e90cdcc1a3-trusted-ca\") pod \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") "
Sep 06 10:31:46.594797 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.594748    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls\") pod \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\" (UID: \"69d7eb68-0175-46cd-9464-19e90cdcc1a3\") "
Sep 06 10:31:46.595206 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.595131    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69d7eb68-0175-46cd-9464-19e90cdcc1a3-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "69d7eb68-0175-46cd-9464-19e90cdcc1a3" (UID: "69d7eb68-0175-46cd-9464-19e90cdcc1a3"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:31:46.595346 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.595208    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "69d7eb68-0175-46cd-9464-19e90cdcc1a3" (UID: "69d7eb68-0175-46cd-9464-19e90cdcc1a3"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:31:46.597135 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.597105    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-kube-api-access-6kkzl" (OuterVolumeSpecName: "kube-api-access-6kkzl") pod "69d7eb68-0175-46cd-9464-19e90cdcc1a3" (UID: "69d7eb68-0175-46cd-9464-19e90cdcc1a3"). InnerVolumeSpecName "kube-api-access-6kkzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:31:46.597135 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.597121    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69d7eb68-0175-46cd-9464-19e90cdcc1a3-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "69d7eb68-0175-46cd-9464-19e90cdcc1a3" (UID: "69d7eb68-0175-46cd-9464-19e90cdcc1a3"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 10:31:46.597354 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.597169    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "69d7eb68-0175-46cd-9464-19e90cdcc1a3" (UID: "69d7eb68-0175-46cd-9464-19e90cdcc1a3"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:31:46.597354 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.597190    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69d7eb68-0175-46cd-9464-19e90cdcc1a3-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "69d7eb68-0175-46cd-9464-19e90cdcc1a3" (UID: "69d7eb68-0175-46cd-9464-19e90cdcc1a3"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 10:31:46.597429 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.597362    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "69d7eb68-0175-46cd-9464-19e90cdcc1a3" (UID: "69d7eb68-0175-46cd-9464-19e90cdcc1a3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:31:46.603396 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.603374    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69d7eb68-0175-46cd-9464-19e90cdcc1a3-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "69d7eb68-0175-46cd-9464-19e90cdcc1a3" (UID: "69d7eb68-0175-46cd-9464-19e90cdcc1a3"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:31:46.695797 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.695771    2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-tls\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:31:46.695797 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.695794    2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-bound-sa-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:31:46.695797 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.695803    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kkzl\" (UniqueName: \"kubernetes.io/projected/69d7eb68-0175-46cd-9464-19e90cdcc1a3-kube-api-access-6kkzl\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:31:46.695992 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.695813    2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69d7eb68-0175-46cd-9464-19e90cdcc1a3-ca-trust-extracted\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:31:46.695992 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.695822    2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69d7eb68-0175-46cd-9464-19e90cdcc1a3-registry-certificates\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:31:46.695992 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.695832    2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69d7eb68-0175-46cd-9464-19e90cdcc1a3-image-registry-private-configuration\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:31:46.695992 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.695841    2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69d7eb68-0175-46cd-9464-19e90cdcc1a3-installation-pull-secrets\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:31:46.695992 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.695848    2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69d7eb68-0175-46cd-9464-19e90cdcc1a3-trusted-ca\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:31:46.732144 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.732117    2557 generic.go:356] "Generic (PLEG): container finished" podID="69d7eb68-0175-46cd-9464-19e90cdcc1a3" containerID="9a6fa712d82dcd82bd882b92a7e5d874f890316170d62afb8f5098b82bb57805" exitCode=0
Sep 06 10:31:46.732261 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.732164    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj"
Sep 06 10:31:46.732261 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.732205    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj" event={"ID":"69d7eb68-0175-46cd-9464-19e90cdcc1a3","Type":"ContainerDied","Data":"9a6fa712d82dcd82bd882b92a7e5d874f890316170d62afb8f5098b82bb57805"}
Sep 06 10:31:46.732261 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.732259    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b78f8f8f-f8nfj" event={"ID":"69d7eb68-0175-46cd-9464-19e90cdcc1a3","Type":"ContainerDied","Data":"b9b0b48285cf7e035559f085bd3399c83d10cea8c6d311573578f6b4b0be94d4"}
Sep 06 10:31:46.732375 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.732285    2557 scope.go:117] "RemoveContainer" containerID="9a6fa712d82dcd82bd882b92a7e5d874f890316170d62afb8f5098b82bb57805"
Sep 06 10:31:46.740873 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.740849    2557 scope.go:117] "RemoveContainer" containerID="9a6fa712d82dcd82bd882b92a7e5d874f890316170d62afb8f5098b82bb57805"
Sep 06 10:31:46.741152 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:31:46.741130    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a6fa712d82dcd82bd882b92a7e5d874f890316170d62afb8f5098b82bb57805\": container with ID starting with 9a6fa712d82dcd82bd882b92a7e5d874f890316170d62afb8f5098b82bb57805 not found: ID does not exist" containerID="9a6fa712d82dcd82bd882b92a7e5d874f890316170d62afb8f5098b82bb57805"
Sep 06 10:31:46.741199 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.741161    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a6fa712d82dcd82bd882b92a7e5d874f890316170d62afb8f5098b82bb57805"} err="failed to get container status \"9a6fa712d82dcd82bd882b92a7e5d874f890316170d62afb8f5098b82bb57805\": rpc error: code = NotFound desc = could not find container \"9a6fa712d82dcd82bd882b92a7e5d874f890316170d62afb8f5098b82bb57805\": container with ID starting with 9a6fa712d82dcd82bd882b92a7e5d874f890316170d62afb8f5098b82bb57805 not found: ID does not exist"
Sep 06 10:31:46.752792 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.752744    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b78f8f8f-f8nfj"]
Sep 06 10:31:46.756329 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:46.756309    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7b78f8f8f-f8nfj"]
Sep 06 10:31:47.738890 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:47.738860    2557 generic.go:356] "Generic (PLEG): container finished" podID="1462951b-ed57-4e82-8a1d-5d5f08f73ee9" containerID="71f6f9b0e8bac2e629abba0ab81ba7470d7a8ee0dcbde3d1f0c876a1841cb32b" exitCode=0
Sep 06 10:31:47.739273 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:47.738934    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-p8s2s" event={"ID":"1462951b-ed57-4e82-8a1d-5d5f08f73ee9","Type":"ContainerDied","Data":"71f6f9b0e8bac2e629abba0ab81ba7470d7a8ee0dcbde3d1f0c876a1841cb32b"}
Sep 06 10:31:47.739273 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:47.739253    2557 scope.go:117] "RemoveContainer" containerID="71f6f9b0e8bac2e629abba0ab81ba7470d7a8ee0dcbde3d1f0c876a1841cb32b"
Sep 06 10:31:48.037382 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:48.037328    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69d7eb68-0175-46cd-9464-19e90cdcc1a3" path="/var/lib/kubelet/pods/69d7eb68-0175-46cd-9464-19e90cdcc1a3/volumes"
Sep 06 10:31:48.744474 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:48.744440    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-p8s2s" event={"ID":"1462951b-ed57-4e82-8a1d-5d5f08f73ee9","Type":"ContainerStarted","Data":"1252663e5f439693b7c74c753d382df181604f544c3a1e90d6cb9086676234b7"}
Sep 06 10:31:52.760364 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:52.760329    2557 generic.go:356] "Generic (PLEG): container finished" podID="679d9837-653d-4ac4-b786-dd55ac8a5e06" containerID="a9ac8581937621eb7e28a4ef5ae74e120766ab4e2eca36bb9b85bb927a5f7ce1" exitCode=0
Sep 06 10:31:52.760769 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:52.760373    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g" event={"ID":"679d9837-653d-4ac4-b786-dd55ac8a5e06","Type":"ContainerDied","Data":"a9ac8581937621eb7e28a4ef5ae74e120766ab4e2eca36bb9b85bb927a5f7ce1"}
Sep 06 10:31:52.760769 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:52.760700    2557 scope.go:117] "RemoveContainer" containerID="a9ac8581937621eb7e28a4ef5ae74e120766ab4e2eca36bb9b85bb927a5f7ce1"
Sep 06 10:31:53.766420 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:31:53.766383    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-vnd8g" event={"ID":"679d9837-653d-4ac4-b786-dd55ac8a5e06","Type":"ContainerStarted","Data":"02a4584ee1e3d8f5d4e98cd2357dac4240299c924b4d17776eb2be822561a3d6"}
Sep 06 10:34:10.880464 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:10.880386    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"]
Sep 06 10:34:10.880953 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:10.880749    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69d7eb68-0175-46cd-9464-19e90cdcc1a3" containerName="registry"
Sep 06 10:34:10.880953 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:10.880763    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="69d7eb68-0175-46cd-9464-19e90cdcc1a3" containerName="registry"
Sep 06 10:34:10.880953 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:10.880821    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="69d7eb68-0175-46cd-9464-19e90cdcc1a3" containerName="registry"
Sep 06 10:34:10.883574 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:10.883558    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"
Sep 06 10:34:10.887104 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:10.887081    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 06 10:34:10.887249 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:10.887146    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 06 10:34:10.887386 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:10.887363    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fg8xs\""
Sep 06 10:34:10.887446 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:10.887376    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 06 10:34:10.887446 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:10.887403    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 06 10:34:10.905204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:10.905180    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"]
Sep 06 10:34:10.930590 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:10.930565    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/015b1b2b-33d7-4c34-9416-bd9a64fdc040-webhook-cert\") pod \"opendatahub-operator-controller-manager-645f664568-rf25s\" (UID: \"015b1b2b-33d7-4c34-9416-bd9a64fdc040\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"
Sep 06 10:34:10.930678 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:10.930592    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/015b1b2b-33d7-4c34-9416-bd9a64fdc040-tmp\") pod \"opendatahub-operator-controller-manager-645f664568-rf25s\" (UID: \"015b1b2b-33d7-4c34-9416-bd9a64fdc040\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"
Sep 06 10:34:10.930678 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:10.930608    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/015b1b2b-33d7-4c34-9416-bd9a64fdc040-manifests\") pod \"opendatahub-operator-controller-manager-645f664568-rf25s\" (UID: \"015b1b2b-33d7-4c34-9416-bd9a64fdc040\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"
Sep 06 10:34:10.930678 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:10.930637    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/015b1b2b-33d7-4c34-9416-bd9a64fdc040-apiservice-cert\") pod \"opendatahub-operator-controller-manager-645f664568-rf25s\" (UID: \"015b1b2b-33d7-4c34-9416-bd9a64fdc040\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"
Sep 06 10:34:10.930789 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:10.930703    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nd7p\" (UniqueName: \"kubernetes.io/projected/015b1b2b-33d7-4c34-9416-bd9a64fdc040-kube-api-access-4nd7p\") pod \"opendatahub-operator-controller-manager-645f664568-rf25s\" (UID: \"015b1b2b-33d7-4c34-9416-bd9a64fdc040\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"
Sep 06 10:34:11.031548 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:11.031521    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nd7p\" (UniqueName: \"kubernetes.io/projected/015b1b2b-33d7-4c34-9416-bd9a64fdc040-kube-api-access-4nd7p\") pod \"opendatahub-operator-controller-manager-645f664568-rf25s\" (UID: \"015b1b2b-33d7-4c34-9416-bd9a64fdc040\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"
Sep 06 10:34:11.031715 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:11.031600    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/015b1b2b-33d7-4c34-9416-bd9a64fdc040-webhook-cert\") pod \"opendatahub-operator-controller-manager-645f664568-rf25s\" (UID: \"015b1b2b-33d7-4c34-9416-bd9a64fdc040\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"
Sep 06 10:34:11.031715 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:11.031628    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/015b1b2b-33d7-4c34-9416-bd9a64fdc040-tmp\") pod \"opendatahub-operator-controller-manager-645f664568-rf25s\" (UID: \"015b1b2b-33d7-4c34-9416-bd9a64fdc040\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"
Sep 06 10:34:11.031715 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:11.031649    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/015b1b2b-33d7-4c34-9416-bd9a64fdc040-manifests\") pod \"opendatahub-operator-controller-manager-645f664568-rf25s\" (UID: \"015b1b2b-33d7-4c34-9416-bd9a64fdc040\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"
Sep 06 10:34:11.031715 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:11.031692    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/015b1b2b-33d7-4c34-9416-bd9a64fdc040-apiservice-cert\") pod \"opendatahub-operator-controller-manager-645f664568-rf25s\" (UID: \"015b1b2b-33d7-4c34-9416-bd9a64fdc040\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"
Sep 06 10:34:11.032076 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:11.032040    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/015b1b2b-33d7-4c34-9416-bd9a64fdc040-manifests\") pod \"opendatahub-operator-controller-manager-645f664568-rf25s\" (UID: \"015b1b2b-33d7-4c34-9416-bd9a64fdc040\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"
Sep 06 10:34:11.034066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:11.034032    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/015b1b2b-33d7-4c34-9416-bd9a64fdc040-tmp\") pod \"opendatahub-operator-controller-manager-645f664568-rf25s\" (UID: \"015b1b2b-33d7-4c34-9416-bd9a64fdc040\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"
Sep 06 10:34:11.034165 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:11.034111    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/015b1b2b-33d7-4c34-9416-bd9a64fdc040-webhook-cert\") pod \"opendatahub-operator-controller-manager-645f664568-rf25s\" (UID: \"015b1b2b-33d7-4c34-9416-bd9a64fdc040\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"
Sep 06 10:34:11.034165 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:11.034135    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/015b1b2b-33d7-4c34-9416-bd9a64fdc040-apiservice-cert\") pod \"opendatahub-operator-controller-manager-645f664568-rf25s\" (UID: \"015b1b2b-33d7-4c34-9416-bd9a64fdc040\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"
Sep 06 10:34:11.042688 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:11.042663    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nd7p\" (UniqueName: \"kubernetes.io/projected/015b1b2b-33d7-4c34-9416-bd9a64fdc040-kube-api-access-4nd7p\") pod \"opendatahub-operator-controller-manager-645f664568-rf25s\" (UID: \"015b1b2b-33d7-4c34-9416-bd9a64fdc040\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"
Sep 06 10:34:11.193846 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:11.193762    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"
Sep 06 10:34:11.471299 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:11.471272    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"]
Sep 06 10:34:11.473609 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:34:11.473577    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod015b1b2b_33d7_4c34_9416_bd9a64fdc040.slice/crio-cf968dc76fdb7b410665a4130c905a8383c985c10263ee12491ab16cb7d7eb1f WatchSource:0}: Error finding container cf968dc76fdb7b410665a4130c905a8383c985c10263ee12491ab16cb7d7eb1f: Status 404 returned error can't find the container with id cf968dc76fdb7b410665a4130c905a8383c985c10263ee12491ab16cb7d7eb1f
Sep 06 10:34:12.291732 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:12.291689    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s" event={"ID":"015b1b2b-33d7-4c34-9416-bd9a64fdc040","Type":"ContainerStarted","Data":"cf968dc76fdb7b410665a4130c905a8383c985c10263ee12491ab16cb7d7eb1f"}
Sep 06 10:34:15.306547 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:15.306513    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s" event={"ID":"015b1b2b-33d7-4c34-9416-bd9a64fdc040","Type":"ContainerStarted","Data":"73a87e992763db51a24a856595b5152bcb81e849f510e1bced80f9b89c037895"}
Sep 06 10:34:16.312614 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:16.312574    2557 generic.go:356] "Generic (PLEG): container finished" podID="015b1b2b-33d7-4c34-9416-bd9a64fdc040" containerID="73a87e992763db51a24a856595b5152bcb81e849f510e1bced80f9b89c037895" exitCode=0
Sep 06 10:34:16.313068 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:16.312618    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s" event={"ID":"015b1b2b-33d7-4c34-9416-bd9a64fdc040","Type":"ContainerDied","Data":"73a87e992763db51a24a856595b5152bcb81e849f510e1bced80f9b89c037895"}
Sep 06 10:34:17.319092 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:17.319062    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s" event={"ID":"015b1b2b-33d7-4c34-9416-bd9a64fdc040","Type":"ContainerStarted","Data":"f14e24d4d9ea081af5e2ef3a5a4806f56d8e2c9feeaeed3da32782005dbbfcef"}
Sep 06 10:34:17.319454 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:17.319171    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"
Sep 06 10:34:17.340481 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:17.340442    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s" podStartSLOduration=3.00251188 podStartE2EDuration="7.340429444s" totalImagesPullingTime="3.253551922s" totalInitContainerRuntime="1.084365642s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:34:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:34:17.339244847 +0000 UTC m=+269.789025614" watchObservedRunningTime="2026-09-06 10:34:17.340429444 +0000 UTC m=+269.790210212"
Sep 06 10:34:28.325007 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:28.324977    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-645f664568-rf25s"
Sep 06 10:34:35.450955 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:35.450915    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5586db6db5-hp98n"]
Sep 06 10:34:35.460948 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:35.460922    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5586db6db5-hp98n"
Sep 06 10:34:35.463325 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:35.463295    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 06 10:34:35.463451 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:35.463342    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 06 10:34:35.465259 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:35.465234    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5586db6db5-hp98n"]
Sep 06 10:34:35.618015 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:35.617984    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e1ab4e3-c725-4ae9-8e94-0f0739e10460-tmp\") pod \"kube-auth-proxy-5586db6db5-hp98n\" (UID: \"9e1ab4e3-c725-4ae9-8e94-0f0739e10460\") " pod="openshift-ingress/kube-auth-proxy-5586db6db5-hp98n"
Sep 06 10:34:35.618178 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:35.618045    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e1ab4e3-c725-4ae9-8e94-0f0739e10460-tls-certs\") pod \"kube-auth-proxy-5586db6db5-hp98n\" (UID: \"9e1ab4e3-c725-4ae9-8e94-0f0739e10460\") " pod="openshift-ingress/kube-auth-proxy-5586db6db5-hp98n"
Sep 06 10:34:35.618178 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:35.618097    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76zp4\" (UniqueName: \"kubernetes.io/projected/9e1ab4e3-c725-4ae9-8e94-0f0739e10460-kube-api-access-76zp4\") pod \"kube-auth-proxy-5586db6db5-hp98n\" (UID: \"9e1ab4e3-c725-4ae9-8e94-0f0739e10460\") " pod="openshift-ingress/kube-auth-proxy-5586db6db5-hp98n"
Sep 06 10:34:35.718790 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:35.718713    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e1ab4e3-c725-4ae9-8e94-0f0739e10460-tmp\") pod \"kube-auth-proxy-5586db6db5-hp98n\" (UID: \"9e1ab4e3-c725-4ae9-8e94-0f0739e10460\") " pod="openshift-ingress/kube-auth-proxy-5586db6db5-hp98n"
Sep 06 10:34:35.718790 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:35.718766    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e1ab4e3-c725-4ae9-8e94-0f0739e10460-tls-certs\") pod \"kube-auth-proxy-5586db6db5-hp98n\" (UID: \"9e1ab4e3-c725-4ae9-8e94-0f0739e10460\") " pod="openshift-ingress/kube-auth-proxy-5586db6db5-hp98n"
Sep 06 10:34:35.718961 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:35.718796    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76zp4\" (UniqueName: \"kubernetes.io/projected/9e1ab4e3-c725-4ae9-8e94-0f0739e10460-kube-api-access-76zp4\") pod \"kube-auth-proxy-5586db6db5-hp98n\" (UID: \"9e1ab4e3-c725-4ae9-8e94-0f0739e10460\") " pod="openshift-ingress/kube-auth-proxy-5586db6db5-hp98n"
Sep 06 10:34:35.721088 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:35.721062    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e1ab4e3-c725-4ae9-8e94-0f0739e10460-tmp\") pod \"kube-auth-proxy-5586db6db5-hp98n\" (UID: \"9e1ab4e3-c725-4ae9-8e94-0f0739e10460\") " pod="openshift-ingress/kube-auth-proxy-5586db6db5-hp98n"
Sep 06 10:34:35.721284 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:35.721264    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e1ab4e3-c725-4ae9-8e94-0f0739e10460-tls-certs\") pod \"kube-auth-proxy-5586db6db5-hp98n\" (UID: \"9e1ab4e3-c725-4ae9-8e94-0f0739e10460\") " pod="openshift-ingress/kube-auth-proxy-5586db6db5-hp98n"
Sep 06 10:34:35.727016 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:35.726989    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76zp4\" (UniqueName: \"kubernetes.io/projected/9e1ab4e3-c725-4ae9-8e94-0f0739e10460-kube-api-access-76zp4\") pod \"kube-auth-proxy-5586db6db5-hp98n\" (UID: \"9e1ab4e3-c725-4ae9-8e94-0f0739e10460\") " pod="openshift-ingress/kube-auth-proxy-5586db6db5-hp98n"
Sep 06 10:34:35.772013 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:35.771982    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5586db6db5-hp98n"
Sep 06 10:34:36.041801 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.041777    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5586db6db5-hp98n"]
Sep 06 10:34:36.044170 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:34:36.044138    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e1ab4e3_c725_4ae9_8e94_0f0739e10460.slice/crio-c610fa72bf6e1f78d408dfde89f7c097792d53b39368a969e1f68212c8f47524 WatchSource:0}: Error finding container c610fa72bf6e1f78d408dfde89f7c097792d53b39368a969e1f68212c8f47524: Status 404 returned error can't find the container with id c610fa72bf6e1f78d408dfde89f7c097792d53b39368a969e1f68212c8f47524
Sep 06 10:34:36.399267 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.399231    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5586db6db5-hp98n" event={"ID":"9e1ab4e3-c725-4ae9-8e94-0f0739e10460","Type":"ContainerStarted","Data":"c610fa72bf6e1f78d408dfde89f7c097792d53b39368a969e1f68212c8f47524"}
Sep 06 10:34:36.679977 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.679891    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"]
Sep 06 10:34:36.682686 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.682660    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.685821 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.685513    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 06 10:34:36.685821 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.685522    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 06 10:34:36.685821 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.685667    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 06 10:34:36.685821 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.685749    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-dv29f\""
Sep 06 10:34:36.686099 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.686070    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 06 10:34:36.695758 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.695727    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"]
Sep 06 10:34:36.827963 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.827911    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.827963 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.827965    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6c5a1df3-22c2-477c-9458-f1e9528778a2-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.828202 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.827999    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8vw8\" (UniqueName: \"kubernetes.io/projected/6c5a1df3-22c2-477c-9458-f1e9528778a2-kube-api-access-s8vw8\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.828202 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.828033    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.828202 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.828061    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c5a1df3-22c2-477c-9458-f1e9528778a2-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.828202 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.828082    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.828202 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.828120    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.929463 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.929423    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.929647 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.929478    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6c5a1df3-22c2-477c-9458-f1e9528778a2-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.929647 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.929539    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8vw8\" (UniqueName: \"kubernetes.io/projected/6c5a1df3-22c2-477c-9458-f1e9528778a2-kube-api-access-s8vw8\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.929647 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.929576    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.929647 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.929604    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c5a1df3-22c2-477c-9458-f1e9528778a2-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.929647 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.929628    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.929970 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.929670    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.931560 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.931415    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.932664 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.932640    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.933001 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.932957    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.933189 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.933167    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6c5a1df3-22c2-477c-9458-f1e9528778a2-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.934168 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.934125    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c5a1df3-22c2-477c-9458-f1e9528778a2-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.938569 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.938447    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8vw8\" (UniqueName: \"kubernetes.io/projected/6c5a1df3-22c2-477c-9458-f1e9528778a2-kube-api-access-s8vw8\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.939371 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.939343    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-6qzwg\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:36.997562 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:36.997530    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:37.284706 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:37.284637    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"]
Sep 06 10:34:37.287662 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:34:37.287636    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c5a1df3_22c2_477c_9458_f1e9528778a2.slice/crio-eb9196342cd5ececcf89dfb5bd25d3ccd8a1656a0986cc403b27551293c56a4c WatchSource:0}: Error finding container eb9196342cd5ececcf89dfb5bd25d3ccd8a1656a0986cc403b27551293c56a4c: Status 404 returned error can't find the container with id eb9196342cd5ececcf89dfb5bd25d3ccd8a1656a0986cc403b27551293c56a4c
Sep 06 10:34:37.406367 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:37.406315    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg" event={"ID":"6c5a1df3-22c2-477c-9458-f1e9528778a2","Type":"ContainerStarted","Data":"eb9196342cd5ececcf89dfb5bd25d3ccd8a1656a0986cc403b27551293c56a4c"}
Sep 06 10:34:39.418379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:39.418346    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5586db6db5-hp98n" event={"ID":"9e1ab4e3-c725-4ae9-8e94-0f0739e10460","Type":"ContainerStarted","Data":"2f1423b5a3d8de9907b7330471fb18c9358de587851013251fcf5cf20b2c4e7b"}
Sep 06 10:34:39.438122 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:39.438074    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5586db6db5-hp98n" podStartSLOduration=1.640175408 podStartE2EDuration="4.438058914s" totalImagesPullingTime="2.797883506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:34:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:34:39.436300344 +0000 UTC m=+291.886081107" watchObservedRunningTime="2026-09-06 10:34:39.438058914 +0000 UTC m=+291.887839682"
Sep 06 10:34:40.605926 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:40.605888    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:34:40.606172 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:40.605957    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:34:41.431294 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:41.431259    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg" event={"ID":"6c5a1df3-22c2-477c-9458-f1e9528778a2","Type":"ContainerStarted","Data":"5dabb6e9f2eec820aa589113f9f6503b0f02b06eaa73a82b6d537e7ef6e53143"}
Sep 06 10:34:41.431559 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:41.431485    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:41.433306 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:41.433276    2557 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-6qzwg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 06 10:34:41.433452 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:41.433336    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg" podUID="6c5a1df3-22c2-477c-9458-f1e9528778a2" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 10:34:41.461529 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:41.461457    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg" podStartSLOduration=2.1458104159999998 podStartE2EDuration="5.461437216s" totalImagesPullingTime="3.3156268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:34:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:34:41.460398247 +0000 UTC m=+293.910179016" watchObservedRunningTime="2026-09-06 10:34:41.461437216 +0000 UTC m=+293.911217986"
Sep 06 10:34:42.436532 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:42.436485    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:47.981249 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:47.981218    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/2.log"
Sep 06 10:34:47.981855 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:47.981830    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/2.log"
Sep 06 10:34:47.993398 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:47.993375    2557 kubelet.go:1648] "Image garbage collection succeeded"
Sep 06 10:34:52.013437 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:52.013403    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wghzk"]
Sep 06 10:34:52.016701 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:52.016686    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wghzk"
Sep 06 10:34:52.019991 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:52.019958    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 06 10:34:52.020600 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:52.020577    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-kpf6n\""
Sep 06 10:34:52.045399 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:52.045376    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wghzk"]
Sep 06 10:34:52.056218 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:52.056193    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54d0d5eb-abf2-46a4-8abf-b39d69b35a21-cert\") pod \"kserve-controller-manager-d6f4b65b8-wghzk\" (UID: \"54d0d5eb-abf2-46a4-8abf-b39d69b35a21\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wghzk"
Sep 06 10:34:52.056338 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:52.056264    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqq7q\" (UniqueName: \"kubernetes.io/projected/54d0d5eb-abf2-46a4-8abf-b39d69b35a21-kube-api-access-dqq7q\") pod \"kserve-controller-manager-d6f4b65b8-wghzk\" (UID: \"54d0d5eb-abf2-46a4-8abf-b39d69b35a21\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wghzk"
Sep 06 10:34:52.157543 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:52.157479    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54d0d5eb-abf2-46a4-8abf-b39d69b35a21-cert\") pod \"kserve-controller-manager-d6f4b65b8-wghzk\" (UID: \"54d0d5eb-abf2-46a4-8abf-b39d69b35a21\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wghzk"
Sep 06 10:34:52.157699 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:52.157562    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqq7q\" (UniqueName: \"kubernetes.io/projected/54d0d5eb-abf2-46a4-8abf-b39d69b35a21-kube-api-access-dqq7q\") pod \"kserve-controller-manager-d6f4b65b8-wghzk\" (UID: \"54d0d5eb-abf2-46a4-8abf-b39d69b35a21\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wghzk"
Sep 06 10:34:52.157699 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:34:52.157629    2557 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 06 10:34:52.157699 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:34:52.157688    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54d0d5eb-abf2-46a4-8abf-b39d69b35a21-cert podName:54d0d5eb-abf2-46a4-8abf-b39d69b35a21 nodeName:}" failed. No retries permitted until 2026-09-06 10:34:52.657671002 +0000 UTC m=+305.107451765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/54d0d5eb-abf2-46a4-8abf-b39d69b35a21-cert") pod "kserve-controller-manager-d6f4b65b8-wghzk" (UID: "54d0d5eb-abf2-46a4-8abf-b39d69b35a21") : secret "kserve-webhook-server-cert" not found
Sep 06 10:34:52.172957 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:52.172935    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqq7q\" (UniqueName: \"kubernetes.io/projected/54d0d5eb-abf2-46a4-8abf-b39d69b35a21-kube-api-access-dqq7q\") pod \"kserve-controller-manager-d6f4b65b8-wghzk\" (UID: \"54d0d5eb-abf2-46a4-8abf-b39d69b35a21\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wghzk"
Sep 06 10:34:52.664947 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:52.664907    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54d0d5eb-abf2-46a4-8abf-b39d69b35a21-cert\") pod \"kserve-controller-manager-d6f4b65b8-wghzk\" (UID: \"54d0d5eb-abf2-46a4-8abf-b39d69b35a21\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wghzk"
Sep 06 10:34:52.667317 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:52.667291    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54d0d5eb-abf2-46a4-8abf-b39d69b35a21-cert\") pod \"kserve-controller-manager-d6f4b65b8-wghzk\" (UID: \"54d0d5eb-abf2-46a4-8abf-b39d69b35a21\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wghzk"
Sep 06 10:34:52.928231 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:52.928154    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wghzk"
Sep 06 10:34:53.210738 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.210712    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wghzk"]
Sep 06 10:34:53.213136 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:34:53.213106    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54d0d5eb_abf2_46a4_8abf_b39d69b35a21.slice/crio-57cfc0010dab379ea3b1dadc23d6f4471831cc76c4d7f7141a5918f8008b725a WatchSource:0}: Error finding container 57cfc0010dab379ea3b1dadc23d6f4471831cc76c4d7f7141a5918f8008b725a: Status 404 returned error can't find the container with id 57cfc0010dab379ea3b1dadc23d6f4471831cc76c4d7f7141a5918f8008b725a
Sep 06 10:34:53.214610 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.214590    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 10:34:53.312631 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.312604    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"]
Sep 06 10:34:53.312844 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.312797    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg" podUID="6c5a1df3-22c2-477c-9458-f1e9528778a2" containerName="discovery" containerID="cri-o://5dabb6e9f2eec820aa589113f9f6503b0f02b06eaa73a82b6d537e7ef6e53143" gracePeriod=30
Sep 06 10:34:53.512072 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.512032    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wghzk" event={"ID":"54d0d5eb-abf2-46a4-8abf-b39d69b35a21","Type":"ContainerStarted","Data":"57cfc0010dab379ea3b1dadc23d6f4471831cc76c4d7f7141a5918f8008b725a"}
Sep 06 10:34:53.514144 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.514085    2557 generic.go:356] "Generic (PLEG): container finished" podID="6c5a1df3-22c2-477c-9458-f1e9528778a2" containerID="5dabb6e9f2eec820aa589113f9f6503b0f02b06eaa73a82b6d537e7ef6e53143" exitCode=0
Sep 06 10:34:53.514276 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.514159    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg" event={"ID":"6c5a1df3-22c2-477c-9458-f1e9528778a2","Type":"ContainerDied","Data":"5dabb6e9f2eec820aa589113f9f6503b0f02b06eaa73a82b6d537e7ef6e53143"}
Sep 06 10:34:53.562238 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.562214    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:53.574027 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.574003    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-csr-dns-cert\") pod \"6c5a1df3-22c2-477c-9458-f1e9528778a2\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") "
Sep 06 10:34:53.574121 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.574033    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8vw8\" (UniqueName: \"kubernetes.io/projected/6c5a1df3-22c2-477c-9458-f1e9528778a2-kube-api-access-s8vw8\") pod \"6c5a1df3-22c2-477c-9458-f1e9528778a2\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") "
Sep 06 10:34:53.574121 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.574054    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6c5a1df3-22c2-477c-9458-f1e9528778a2-cacerts\") pod \"6c5a1df3-22c2-477c-9458-f1e9528778a2\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") "
Sep 06 10:34:53.574121 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.574072    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-kubeconfig\") pod \"6c5a1df3-22c2-477c-9458-f1e9528778a2\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") "
Sep 06 10:34:53.574280 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.574185    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c5a1df3-22c2-477c-9458-f1e9528778a2-local-certs\") pod \"6c5a1df3-22c2-477c-9458-f1e9528778a2\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") "
Sep 06 10:34:53.574280 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.574247    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-token\") pod \"6c5a1df3-22c2-477c-9458-f1e9528778a2\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") "
Sep 06 10:34:53.574394 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.574277    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-csr-ca-configmap\") pod \"6c5a1df3-22c2-477c-9458-f1e9528778a2\" (UID: \"6c5a1df3-22c2-477c-9458-f1e9528778a2\") "
Sep 06 10:34:53.574905 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.574849    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "6c5a1df3-22c2-477c-9458-f1e9528778a2" (UID: "6c5a1df3-22c2-477c-9458-f1e9528778a2"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:34:53.576794 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.576763    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c5a1df3-22c2-477c-9458-f1e9528778a2-local-certs" (OuterVolumeSpecName: "local-certs") pod "6c5a1df3-22c2-477c-9458-f1e9528778a2" (UID: "6c5a1df3-22c2-477c-9458-f1e9528778a2"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:34:53.576902 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.576816    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "6c5a1df3-22c2-477c-9458-f1e9528778a2" (UID: "6c5a1df3-22c2-477c-9458-f1e9528778a2"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 10:34:53.576902 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.576870    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c5a1df3-22c2-477c-9458-f1e9528778a2-kube-api-access-s8vw8" (OuterVolumeSpecName: "kube-api-access-s8vw8") pod "6c5a1df3-22c2-477c-9458-f1e9528778a2" (UID: "6c5a1df3-22c2-477c-9458-f1e9528778a2"). InnerVolumeSpecName "kube-api-access-s8vw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:34:53.576902 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.576868    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c5a1df3-22c2-477c-9458-f1e9528778a2-cacerts" (OuterVolumeSpecName: "cacerts") pod "6c5a1df3-22c2-477c-9458-f1e9528778a2" (UID: "6c5a1df3-22c2-477c-9458-f1e9528778a2"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 10:34:53.577057 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.576953    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "6c5a1df3-22c2-477c-9458-f1e9528778a2" (UID: "6c5a1df3-22c2-477c-9458-f1e9528778a2"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 10:34:53.577057 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.577032    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-token" (OuterVolumeSpecName: "istio-token") pod "6c5a1df3-22c2-477c-9458-f1e9528778a2" (UID: "6c5a1df3-22c2-477c-9458-f1e9528778a2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:34:53.675374 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.675347    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:34:53.675374 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.675370    2557 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-csr-ca-configmap\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:34:53.675374 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.675379    2557 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-csr-dns-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:34:53.675584 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.675389    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8vw8\" (UniqueName: \"kubernetes.io/projected/6c5a1df3-22c2-477c-9458-f1e9528778a2-kube-api-access-s8vw8\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:34:53.675584 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.675397    2557 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6c5a1df3-22c2-477c-9458-f1e9528778a2-cacerts\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:34:53.675584 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.675406    2557 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6c5a1df3-22c2-477c-9458-f1e9528778a2-istio-kubeconfig\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:34:53.675584 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:53.675414    2557 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c5a1df3-22c2-477c-9458-f1e9528778a2-local-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:34:54.520013 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:54.519976    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg" event={"ID":"6c5a1df3-22c2-477c-9458-f1e9528778a2","Type":"ContainerDied","Data":"eb9196342cd5ececcf89dfb5bd25d3ccd8a1656a0986cc403b27551293c56a4c"}
Sep 06 10:34:54.520426 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:54.520020    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"
Sep 06 10:34:54.520426 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:54.520042    2557 scope.go:117] "RemoveContainer" containerID="5dabb6e9f2eec820aa589113f9f6503b0f02b06eaa73a82b6d537e7ef6e53143"
Sep 06 10:34:54.544772 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:54.544739    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"]
Sep 06 10:34:54.549381 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:54.549358    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6qzwg"]
Sep 06 10:34:55.526151 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:55.526053    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wghzk" event={"ID":"54d0d5eb-abf2-46a4-8abf-b39d69b35a21","Type":"ContainerStarted","Data":"2b970ce4ee7a19f8d77289f21384b37152c200bb608b2a6c6ebccab4bb2625e0"}
Sep 06 10:34:55.526613 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:55.526156    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wghzk"
Sep 06 10:34:55.550430 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:55.550385    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wghzk" podStartSLOduration=2.550492968 podStartE2EDuration="4.550370489s" totalImagesPullingTime="1.999877521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:34:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:34:55.548480374 +0000 UTC m=+307.998261144" watchObservedRunningTime="2026-09-06 10:34:55.550370489 +0000 UTC m=+308.000151256"
Sep 06 10:34:56.035633 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:34:56.035596    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c5a1df3-22c2-477c-9458-f1e9528778a2" path="/var/lib/kubelet/pods/6c5a1df3-22c2-477c-9458-f1e9528778a2/volumes"
Sep 06 10:35:26.536362 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:26.536328    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wghzk"
Sep 06 10:35:57.277639 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:57.277606    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-kj7ht"]
Sep 06 10:35:57.278149 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:57.277991    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c5a1df3-22c2-477c-9458-f1e9528778a2" containerName="discovery"
Sep 06 10:35:57.278149 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:57.278010    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c5a1df3-22c2-477c-9458-f1e9528778a2" containerName="discovery"
Sep 06 10:35:57.278149 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:57.278089    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c5a1df3-22c2-477c-9458-f1e9528778a2" containerName="discovery"
Sep 06 10:35:57.282084 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:57.282066    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-kj7ht"
Sep 06 10:35:57.284540 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:57.284512    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 06 10:35:57.284649 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:57.284547    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-kjpw7\""
Sep 06 10:35:57.284823 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:57.284809    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 06 10:35:57.285365 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:57.285350    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 06 10:35:57.292404 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:57.292380    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-kj7ht"]
Sep 06 10:35:57.450083 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:57.450040    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd79m\" (UniqueName: \"kubernetes.io/projected/f1e5fbce-78b9-44cc-a76a-dc64c25c0780-kube-api-access-qd79m\") pod \"dns-operator-controller-manager-5985564776-kj7ht\" (UID: \"f1e5fbce-78b9-44cc-a76a-dc64c25c0780\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-kj7ht"
Sep 06 10:35:57.551297 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:57.551216    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd79m\" (UniqueName: \"kubernetes.io/projected/f1e5fbce-78b9-44cc-a76a-dc64c25c0780-kube-api-access-qd79m\") pod \"dns-operator-controller-manager-5985564776-kj7ht\" (UID: \"f1e5fbce-78b9-44cc-a76a-dc64c25c0780\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-kj7ht"
Sep 06 10:35:57.559923 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:57.559898    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd79m\" (UniqueName: \"kubernetes.io/projected/f1e5fbce-78b9-44cc-a76a-dc64c25c0780-kube-api-access-qd79m\") pod \"dns-operator-controller-manager-5985564776-kj7ht\" (UID: \"f1e5fbce-78b9-44cc-a76a-dc64c25c0780\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-kj7ht"
Sep 06 10:35:57.592480 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:57.592455    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-kj7ht"
Sep 06 10:35:57.861587 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:57.861504    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-kj7ht"]
Sep 06 10:35:57.864282 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:35:57.864257    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1e5fbce_78b9_44cc_a76a_dc64c25c0780.slice/crio-416ea7303738f9b6a2bc174745eb08e09b1068259758179164e0214637a21aca WatchSource:0}: Error finding container 416ea7303738f9b6a2bc174745eb08e09b1068259758179164e0214637a21aca: Status 404 returned error can't find the container with id 416ea7303738f9b6a2bc174745eb08e09b1068259758179164e0214637a21aca
Sep 06 10:35:58.795438 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:58.795405    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-kj7ht" event={"ID":"f1e5fbce-78b9-44cc-a76a-dc64c25c0780","Type":"ContainerStarted","Data":"416ea7303738f9b6a2bc174745eb08e09b1068259758179164e0214637a21aca"}
Sep 06 10:35:59.478833 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:59.478797    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"]
Sep 06 10:35:59.483130 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:59.483110    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"
Sep 06 10:35:59.485814 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:59.485797    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-lsvrw\""
Sep 06 10:35:59.501268 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:59.501243    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"]
Sep 06 10:35:59.669073 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:59.669043    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcd9j\" (UniqueName: \"kubernetes.io/projected/6d80ae13-b32e-4bed-b781-75eaae79a209-kube-api-access-pcd9j\") pod \"limitador-operator-controller-manager-6c549b6d85-qqwl9\" (UID: \"6d80ae13-b32e-4bed-b781-75eaae79a209\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"
Sep 06 10:35:59.770388 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:59.770330    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcd9j\" (UniqueName: \"kubernetes.io/projected/6d80ae13-b32e-4bed-b781-75eaae79a209-kube-api-access-pcd9j\") pod \"limitador-operator-controller-manager-6c549b6d85-qqwl9\" (UID: \"6d80ae13-b32e-4bed-b781-75eaae79a209\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"
Sep 06 10:35:59.783950 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:59.783924    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcd9j\" (UniqueName: \"kubernetes.io/projected/6d80ae13-b32e-4bed-b781-75eaae79a209-kube-api-access-pcd9j\") pod \"limitador-operator-controller-manager-6c549b6d85-qqwl9\" (UID: \"6d80ae13-b32e-4bed-b781-75eaae79a209\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"
Sep 06 10:35:59.793562 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:35:59.793539    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"
Sep 06 10:36:00.070433 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:00.070410    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"]
Sep 06 10:36:00.072942 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:36:00.072915    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d80ae13_b32e_4bed_b781_75eaae79a209.slice/crio-bd5c3abcaa248436105b3806f9d4ee1be56ccef3ae5b02276427d08f547f57d0 WatchSource:0}: Error finding container bd5c3abcaa248436105b3806f9d4ee1be56ccef3ae5b02276427d08f547f57d0: Status 404 returned error can't find the container with id bd5c3abcaa248436105b3806f9d4ee1be56ccef3ae5b02276427d08f547f57d0
Sep 06 10:36:00.807771 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:00.807730    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9" event={"ID":"6d80ae13-b32e-4bed-b781-75eaae79a209","Type":"ContainerStarted","Data":"bd5c3abcaa248436105b3806f9d4ee1be56ccef3ae5b02276427d08f547f57d0"}
Sep 06 10:36:02.039223 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:02.039193    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-zg5p8"]
Sep 06 10:36:02.044635 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:02.044613    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-zg5p8"
Sep 06 10:36:02.047173 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:02.047147    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-h5r84\""
Sep 06 10:36:02.050701 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:02.050675    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-zg5p8"]
Sep 06 10:36:02.088998 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:02.088971    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xrrq\" (UniqueName: \"kubernetes.io/projected/03d6634e-1105-4bf0-abf2-70c79c9e15d1-kube-api-access-8xrrq\") pod \"authorino-operator-68d88948d8-zg5p8\" (UID: \"03d6634e-1105-4bf0-abf2-70c79c9e15d1\") " pod="kuadrant-system/authorino-operator-68d88948d8-zg5p8"
Sep 06 10:36:02.189842 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:02.189787    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xrrq\" (UniqueName: \"kubernetes.io/projected/03d6634e-1105-4bf0-abf2-70c79c9e15d1-kube-api-access-8xrrq\") pod \"authorino-operator-68d88948d8-zg5p8\" (UID: \"03d6634e-1105-4bf0-abf2-70c79c9e15d1\") " pod="kuadrant-system/authorino-operator-68d88948d8-zg5p8"
Sep 06 10:36:02.197953 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:02.197927    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xrrq\" (UniqueName: \"kubernetes.io/projected/03d6634e-1105-4bf0-abf2-70c79c9e15d1-kube-api-access-8xrrq\") pod \"authorino-operator-68d88948d8-zg5p8\" (UID: \"03d6634e-1105-4bf0-abf2-70c79c9e15d1\") " pod="kuadrant-system/authorino-operator-68d88948d8-zg5p8"
Sep 06 10:36:02.357501 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:02.357451    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-zg5p8"
Sep 06 10:36:03.231642 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:03.231619    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-zg5p8"]
Sep 06 10:36:03.234126 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:36:03.234099    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03d6634e_1105_4bf0_abf2_70c79c9e15d1.slice/crio-44f7f24bf0ccaa37f994927ca539baef4c7144b83a2e48832ecbc7a90595f087 WatchSource:0}: Error finding container 44f7f24bf0ccaa37f994927ca539baef4c7144b83a2e48832ecbc7a90595f087: Status 404 returned error can't find the container with id 44f7f24bf0ccaa37f994927ca539baef4c7144b83a2e48832ecbc7a90595f087
Sep 06 10:36:03.826833 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:03.826800    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-kj7ht" event={"ID":"f1e5fbce-78b9-44cc-a76a-dc64c25c0780","Type":"ContainerStarted","Data":"95ba9be6fe424c1bce99a34b912505ac2eccd43cbe8585fa42ba7de1e9e0b581"}
Sep 06 10:36:03.827002 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:03.826928    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-kj7ht"
Sep 06 10:36:03.828648 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:03.828624    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9" event={"ID":"6d80ae13-b32e-4bed-b781-75eaae79a209","Type":"ContainerStarted","Data":"ba0825000e4cb3b6fbf9d0d80206f97f5874ce267279b6fdee22cbd53ecf4230"}
Sep 06 10:36:03.828785 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:03.828712    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"
Sep 06 10:36:03.830010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:03.829989    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-zg5p8" event={"ID":"03d6634e-1105-4bf0-abf2-70c79c9e15d1","Type":"ContainerStarted","Data":"44f7f24bf0ccaa37f994927ca539baef4c7144b83a2e48832ecbc7a90595f087"}
Sep 06 10:36:03.843708 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:03.843670    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-kj7ht" podStartSLOduration=1.7686330369999999 podStartE2EDuration="6.843658173s" totalImagesPullingTime="5.075025136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:35:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:36:03.84285965 +0000 UTC m=+376.292640442" watchObservedRunningTime="2026-09-06 10:36:03.843658173 +0000 UTC m=+376.293438941"
Sep 06 10:36:03.860563 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:03.860527    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9" podStartSLOduration=1.9878671959999998 podStartE2EDuration="4.860514263s" totalImagesPullingTime="2.872647067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:35:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:36:03.858960574 +0000 UTC m=+376.308741353" watchObservedRunningTime="2026-09-06 10:36:03.860514263 +0000 UTC m=+376.310295031"
Sep 06 10:36:04.836799 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:04.836769    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-zg5p8" event={"ID":"03d6634e-1105-4bf0-abf2-70c79c9e15d1","Type":"ContainerStarted","Data":"35977f6bf9fffcbf8e8085756fe1dbf7257c95b087740533f0dfc8cf12c8012c"}
Sep 06 10:36:04.837181 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:04.836916    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-zg5p8"
Sep 06 10:36:04.854788 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:04.854746    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-zg5p8" podStartSLOduration=1.450690489 podStartE2EDuration="2.854732814s" totalImagesPullingTime="1.404042325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:36:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:36:04.853795834 +0000 UTC m=+377.303576611" watchObservedRunningTime="2026-09-06 10:36:04.854732814 +0000 UTC m=+377.304513582"
Sep 06 10:36:06.529981 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:06.529944    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"]
Sep 06 10:36:06.533296 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:06.533270    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:06.535971 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:06.535947    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8lm44\""
Sep 06 10:36:06.548785 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:06.548763    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"]
Sep 06 10:36:06.627058 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:06.627027    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2csh\" (UniqueName: \"kubernetes.io/projected/65a3beb6-121f-4fba-81af-cb5c73bb3b66-kube-api-access-s2csh\") pod \"kuadrant-operator-controller-manager-58fc56757c-9kr76\" (UID: \"65a3beb6-121f-4fba-81af-cb5c73bb3b66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:06.627058 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:06.627061    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/65a3beb6-121f-4fba-81af-cb5c73bb3b66-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-9kr76\" (UID: \"65a3beb6-121f-4fba-81af-cb5c73bb3b66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:06.728336 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:06.728305    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2csh\" (UniqueName: \"kubernetes.io/projected/65a3beb6-121f-4fba-81af-cb5c73bb3b66-kube-api-access-s2csh\") pod \"kuadrant-operator-controller-manager-58fc56757c-9kr76\" (UID: \"65a3beb6-121f-4fba-81af-cb5c73bb3b66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:06.728422 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:06.728342    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/65a3beb6-121f-4fba-81af-cb5c73bb3b66-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-9kr76\" (UID: \"65a3beb6-121f-4fba-81af-cb5c73bb3b66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:06.728689 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:06.728666    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/65a3beb6-121f-4fba-81af-cb5c73bb3b66-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-9kr76\" (UID: \"65a3beb6-121f-4fba-81af-cb5c73bb3b66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:06.736521 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:06.736483    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2csh\" (UniqueName: \"kubernetes.io/projected/65a3beb6-121f-4fba-81af-cb5c73bb3b66-kube-api-access-s2csh\") pod \"kuadrant-operator-controller-manager-58fc56757c-9kr76\" (UID: \"65a3beb6-121f-4fba-81af-cb5c73bb3b66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:06.842962 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:06.842937    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:07.116263 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:07.116228    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"]
Sep 06 10:36:07.118484 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:36:07.118445    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65a3beb6_121f_4fba_81af_cb5c73bb3b66.slice/crio-9965fcc62d3003b8ab098d5618de805a9a03eb7407e79a81b744e92bcb5df95a WatchSource:0}: Error finding container 9965fcc62d3003b8ab098d5618de805a9a03eb7407e79a81b744e92bcb5df95a: Status 404 returned error can't find the container with id 9965fcc62d3003b8ab098d5618de805a9a03eb7407e79a81b744e92bcb5df95a
Sep 06 10:36:07.852940 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:07.852910    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76" event={"ID":"65a3beb6-121f-4fba-81af-cb5c73bb3b66","Type":"ContainerStarted","Data":"9965fcc62d3003b8ab098d5618de805a9a03eb7407e79a81b744e92bcb5df95a"}
Sep 06 10:36:11.873861 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:11.873820    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76" event={"ID":"65a3beb6-121f-4fba-81af-cb5c73bb3b66","Type":"ContainerStarted","Data":"711e5fdcb9bd61bdf4de2d3e3398ae756229c1aef94832eb8dd0fbac8e0eccef"}
Sep 06 10:36:11.874318 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:11.873919    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:11.895546 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:11.895467    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76" podStartSLOduration=1.774162423 podStartE2EDuration="5.895448629s" totalImagesPullingTime="4.121286206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:36:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:36:11.893701592 +0000 UTC m=+384.343482360" watchObservedRunningTime="2026-09-06 10:36:11.895448629 +0000 UTC m=+384.345229407"
Sep 06 10:36:14.838595 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:14.838567    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-kj7ht"
Sep 06 10:36:14.839025 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:14.838637    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"
Sep 06 10:36:15.843880 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:15.843851    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-zg5p8"
Sep 06 10:36:22.880904 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:22.880870    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:23.908133 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:23.908097    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"]
Sep 06 10:36:23.911805 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:23.911783    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:23.934272 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:23.934251    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"]
Sep 06 10:36:23.966800 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:23.966776    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f390f7c-1af3-4b6a-9212-552cc4bc7aa7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-gmfnm\" (UID: \"8f390f7c-1af3-4b6a-9212-552cc4bc7aa7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:23.966906 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:23.966836    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wjh6\" (UniqueName: \"kubernetes.io/projected/8f390f7c-1af3-4b6a-9212-552cc4bc7aa7-kube-api-access-5wjh6\") pod \"kuadrant-operator-controller-manager-96db98c47-gmfnm\" (UID: \"8f390f7c-1af3-4b6a-9212-552cc4bc7aa7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:24.067909 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.067885    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wjh6\" (UniqueName: \"kubernetes.io/projected/8f390f7c-1af3-4b6a-9212-552cc4bc7aa7-kube-api-access-5wjh6\") pod \"kuadrant-operator-controller-manager-96db98c47-gmfnm\" (UID: \"8f390f7c-1af3-4b6a-9212-552cc4bc7aa7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:24.068030 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.067938    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f390f7c-1af3-4b6a-9212-552cc4bc7aa7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-gmfnm\" (UID: \"8f390f7c-1af3-4b6a-9212-552cc4bc7aa7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:24.068249 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.068233    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f390f7c-1af3-4b6a-9212-552cc4bc7aa7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-gmfnm\" (UID: \"8f390f7c-1af3-4b6a-9212-552cc4bc7aa7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:24.076130 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.076102    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wjh6\" (UniqueName: \"kubernetes.io/projected/8f390f7c-1af3-4b6a-9212-552cc4bc7aa7-kube-api-access-5wjh6\") pod \"kuadrant-operator-controller-manager-96db98c47-gmfnm\" (UID: \"8f390f7c-1af3-4b6a-9212-552cc4bc7aa7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:24.221472 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.221419    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:24.494927 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.494903    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"]
Sep 06 10:36:24.497158 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:36:24.497130    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f390f7c_1af3_4b6a_9212_552cc4bc7aa7.slice/crio-33fec64301cfe10a516b663579a00f3a80767da330007e6f031afa5586c578ec WatchSource:0}: Error finding container 33fec64301cfe10a516b663579a00f3a80767da330007e6f031afa5586c578ec: Status 404 returned error can't find the container with id 33fec64301cfe10a516b663579a00f3a80767da330007e6f031afa5586c578ec
Sep 06 10:36:24.883859 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.883830    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"]
Sep 06 10:36:24.884022 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.884006    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76" podUID="65a3beb6-121f-4fba-81af-cb5c73bb3b66" containerName="manager" containerID="cri-o://711e5fdcb9bd61bdf4de2d3e3398ae756229c1aef94832eb8dd0fbac8e0eccef" gracePeriod=2
Sep 06 10:36:24.901394 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.901368    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"]
Sep 06 10:36:24.901680 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:24.901645    2557 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"65a3beb6-121f-4fba-81af-cb5c73bb3b66\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-58fc56757c-9kr76\": pods \"kuadrant-operator-controller-manager-58fc56757c-9kr76\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:24.913144 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.913121    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"]
Sep 06 10:36:24.918073 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.918050    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"]
Sep 06 10:36:24.929515 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.929480    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"]
Sep 06 10:36:24.929702 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.929677    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9" podUID="6d80ae13-b32e-4bed-b781-75eaae79a209" containerName="manager" containerID="cri-o://ba0825000e4cb3b6fbf9d0d80206f97f5874ce267279b6fdee22cbd53ecf4230" gracePeriod=2
Sep 06 10:36:24.943200 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.942792    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n"]
Sep 06 10:36:24.943336 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.943284    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65a3beb6-121f-4fba-81af-cb5c73bb3b66" containerName="manager"
Sep 06 10:36:24.943336 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.943303    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="65a3beb6-121f-4fba-81af-cb5c73bb3b66" containerName="manager"
Sep 06 10:36:24.943336 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.943328    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f390f7c-1af3-4b6a-9212-552cc4bc7aa7" containerName="manager"
Sep 06 10:36:24.943598 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.943338    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f390f7c-1af3-4b6a-9212-552cc4bc7aa7" containerName="manager"
Sep 06 10:36:24.943598 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.943424    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="65a3beb6-121f-4fba-81af-cb5c73bb3b66" containerName="manager"
Sep 06 10:36:24.943598 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.943439    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f390f7c-1af3-4b6a-9212-552cc4bc7aa7" containerName="manager"
Sep 06 10:36:24.946700 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:24.946670    2557 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm" container="manager"
Sep 06 10:36:24.946835 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.946815    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"]
Sep 06 10:36:24.946930 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.946916    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n"
Sep 06 10:36:24.947362 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:24.947328    2557 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"6d80ae13-b32e-4bed-b781-75eaae79a209\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-6c549b6d85-qqwl9\": pods \"limitador-operator-controller-manager-6c549b6d85-qqwl9\" not found" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"
Sep 06 10:36:24.949148 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:24.949120    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-9kr76\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="65a3beb6-121f-4fba-81af-cb5c73bb3b66" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:24.950988 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:24.950960    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-qqwl9\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="6d80ae13-b32e-4bed-b781-75eaae79a209" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"
Sep 06 10:36:24.953018 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:24.952991    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-9kr76\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="65a3beb6-121f-4fba-81af-cb5c73bb3b66" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:24.953350 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.953328    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jwtd2"]
Sep 06 10:36:24.953633 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.953619    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d80ae13-b32e-4bed-b781-75eaae79a209" containerName="manager"
Sep 06 10:36:24.953633 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.953633    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d80ae13-b32e-4bed-b781-75eaae79a209" containerName="manager"
Sep 06 10:36:24.953767 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.953699    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d80ae13-b32e-4bed-b781-75eaae79a209" containerName="manager"
Sep 06 10:36:24.954909 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:24.954886    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-gmfnm\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="8f390f7c-1af3-4b6a-9212-552cc4bc7aa7" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:24.956600 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.956564    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jwtd2"
Sep 06 10:36:24.959593 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.959573    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n"]
Sep 06 10:36:24.968372 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:24.968350    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jwtd2"]
Sep 06 10:36:24.993757 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:24.993727    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-qqwl9\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="6d80ae13-b32e-4bed-b781-75eaae79a209" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"
Sep 06 10:36:24.995620 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:24.995598    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-9kr76\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="65a3beb6-121f-4fba-81af-cb5c73bb3b66" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:24.997260 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:24.997232    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-gmfnm\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="8f390f7c-1af3-4b6a-9212-552cc4bc7aa7" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:25.085894 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.083948    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f109c05-a348-4f2f-9124-c424af9b39de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-bzm9n\" (UID: \"7f109c05-a348-4f2f-9124-c424af9b39de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n"
Sep 06 10:36:25.085894 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.084076    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm4hn\" (UniqueName: \"kubernetes.io/projected/befe56cc-b780-45e3-ae3f-39dc7988bc93-kube-api-access-sm4hn\") pod \"limitador-operator-controller-manager-6c549b6d85-jwtd2\" (UID: \"befe56cc-b780-45e3-ae3f-39dc7988bc93\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jwtd2"
Sep 06 10:36:25.085894 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.084149    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gp6p\" (UniqueName: \"kubernetes.io/projected/7f109c05-a348-4f2f-9124-c424af9b39de-kube-api-access-4gp6p\") pod \"kuadrant-operator-controller-manager-96db98c47-bzm9n\" (UID: \"7f109c05-a348-4f2f-9124-c424af9b39de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n"
Sep 06 10:36:25.178623 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.178598    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:25.180970 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:25.180944    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-qqwl9\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="6d80ae13-b32e-4bed-b781-75eaae79a209" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"
Sep 06 10:36:25.181065 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.181015    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"
Sep 06 10:36:25.182897 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:25.182875    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-9kr76\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="65a3beb6-121f-4fba-81af-cb5c73bb3b66" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:25.184712 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:25.184690    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-gmfnm\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="8f390f7c-1af3-4b6a-9212-552cc4bc7aa7" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:25.184886 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.184870    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gp6p\" (UniqueName: \"kubernetes.io/projected/7f109c05-a348-4f2f-9124-c424af9b39de-kube-api-access-4gp6p\") pod \"kuadrant-operator-controller-manager-96db98c47-bzm9n\" (UID: \"7f109c05-a348-4f2f-9124-c424af9b39de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n"
Sep 06 10:36:25.184944 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.184929    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f109c05-a348-4f2f-9124-c424af9b39de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-bzm9n\" (UID: \"7f109c05-a348-4f2f-9124-c424af9b39de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n"
Sep 06 10:36:25.185098 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.185082    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm4hn\" (UniqueName: \"kubernetes.io/projected/befe56cc-b780-45e3-ae3f-39dc7988bc93-kube-api-access-sm4hn\") pod \"limitador-operator-controller-manager-6c549b6d85-jwtd2\" (UID: \"befe56cc-b780-45e3-ae3f-39dc7988bc93\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jwtd2"
Sep 06 10:36:25.185283 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.185266    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f109c05-a348-4f2f-9124-c424af9b39de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-bzm9n\" (UID: \"7f109c05-a348-4f2f-9124-c424af9b39de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n"
Sep 06 10:36:25.186410 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:25.186388    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-gmfnm\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="8f390f7c-1af3-4b6a-9212-552cc4bc7aa7" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:25.187878 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:25.187860    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-qqwl9\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="6d80ae13-b32e-4bed-b781-75eaae79a209" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"
Sep 06 10:36:25.189562 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:25.189539    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-9kr76\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="65a3beb6-121f-4fba-81af-cb5c73bb3b66" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:25.194430 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.194409    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gp6p\" (UniqueName: \"kubernetes.io/projected/7f109c05-a348-4f2f-9124-c424af9b39de-kube-api-access-4gp6p\") pod \"kuadrant-operator-controller-manager-96db98c47-bzm9n\" (UID: \"7f109c05-a348-4f2f-9124-c424af9b39de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n"
Sep 06 10:36:25.194609 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.194593    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm4hn\" (UniqueName: \"kubernetes.io/projected/befe56cc-b780-45e3-ae3f-39dc7988bc93-kube-api-access-sm4hn\") pod \"limitador-operator-controller-manager-6c549b6d85-jwtd2\" (UID: \"befe56cc-b780-45e3-ae3f-39dc7988bc93\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jwtd2"
Sep 06 10:36:25.285474 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.285450    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2csh\" (UniqueName: \"kubernetes.io/projected/65a3beb6-121f-4fba-81af-cb5c73bb3b66-kube-api-access-s2csh\") pod \"65a3beb6-121f-4fba-81af-cb5c73bb3b66\" (UID: \"65a3beb6-121f-4fba-81af-cb5c73bb3b66\") "
Sep 06 10:36:25.285583 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.285523    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcd9j\" (UniqueName: \"kubernetes.io/projected/6d80ae13-b32e-4bed-b781-75eaae79a209-kube-api-access-pcd9j\") pod \"6d80ae13-b32e-4bed-b781-75eaae79a209\" (UID: \"6d80ae13-b32e-4bed-b781-75eaae79a209\") "
Sep 06 10:36:25.285583 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.285546    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/65a3beb6-121f-4fba-81af-cb5c73bb3b66-extensions-socket-volume\") pod \"65a3beb6-121f-4fba-81af-cb5c73bb3b66\" (UID: \"65a3beb6-121f-4fba-81af-cb5c73bb3b66\") "
Sep 06 10:36:25.286057 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.286032    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65a3beb6-121f-4fba-81af-cb5c73bb3b66-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "65a3beb6-121f-4fba-81af-cb5c73bb3b66" (UID: "65a3beb6-121f-4fba-81af-cb5c73bb3b66"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:36:25.287325 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.287296    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65a3beb6-121f-4fba-81af-cb5c73bb3b66-kube-api-access-s2csh" (OuterVolumeSpecName: "kube-api-access-s2csh") pod "65a3beb6-121f-4fba-81af-cb5c73bb3b66" (UID: "65a3beb6-121f-4fba-81af-cb5c73bb3b66"). InnerVolumeSpecName "kube-api-access-s2csh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:36:25.287407 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.287321    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d80ae13-b32e-4bed-b781-75eaae79a209-kube-api-access-pcd9j" (OuterVolumeSpecName: "kube-api-access-pcd9j") pod "6d80ae13-b32e-4bed-b781-75eaae79a209" (UID: "6d80ae13-b32e-4bed-b781-75eaae79a209"). InnerVolumeSpecName "kube-api-access-pcd9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:36:25.306498 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.306472    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n"
Sep 06 10:36:25.311248 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.311225    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jwtd2"
Sep 06 10:36:25.386903 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.386869    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2csh\" (UniqueName: \"kubernetes.io/projected/65a3beb6-121f-4fba-81af-cb5c73bb3b66-kube-api-access-s2csh\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:36:25.387001 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.386910    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcd9j\" (UniqueName: \"kubernetes.io/projected/6d80ae13-b32e-4bed-b781-75eaae79a209-kube-api-access-pcd9j\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:36:25.387001 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.386925    2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/65a3beb6-121f-4fba-81af-cb5c73bb3b66-extensions-socket-volume\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:36:25.585996 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.585906    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jwtd2"]
Sep 06 10:36:25.587839 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.587813    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n"]
Sep 06 10:36:25.588903 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:36:25.588873    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbefe56cc_b780_45e3_ae3f_39dc7988bc93.slice/crio-362590d85e1885baf83e5c27c807addabedada83702def365a0024e2408c5065 WatchSource:0}: Error finding container 362590d85e1885baf83e5c27c807addabedada83702def365a0024e2408c5065: Status 404 returned error can't find the container with id 362590d85e1885baf83e5c27c807addabedada83702def365a0024e2408c5065
Sep 06 10:36:25.589874 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:36:25.589854    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f109c05_a348_4f2f_9124_c424af9b39de.slice/crio-5935087463bddb0c1f014cc685283b6d990645b19ebf2b27da4b917c1a524220 WatchSource:0}: Error finding container 5935087463bddb0c1f014cc685283b6d990645b19ebf2b27da4b917c1a524220: Status 404 returned error can't find the container with id 5935087463bddb0c1f014cc685283b6d990645b19ebf2b27da4b917c1a524220
Sep 06 10:36:25.953399 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.953358    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm" podUID="8f390f7c-1af3-4b6a-9212-552cc4bc7aa7" containerName="manager" containerID="cri-o://e2f7cc73620fb6706467f5075b5e86915e650dfc06a5e87434f96cbf70b6eea9" gracePeriod=2
Sep 06 10:36:25.954985 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.954955    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jwtd2" event={"ID":"befe56cc-b780-45e3-ae3f-39dc7988bc93","Type":"ContainerStarted","Data":"90b164126350e8f5d3c46f681b771755aba38b5500bc3866bf6b9d9bd11bc9fd"}
Sep 06 10:36:25.955092 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.955010    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jwtd2" event={"ID":"befe56cc-b780-45e3-ae3f-39dc7988bc93","Type":"ContainerStarted","Data":"362590d85e1885baf83e5c27c807addabedada83702def365a0024e2408c5065"}
Sep 06 10:36:25.955092 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.955056    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jwtd2"
Sep 06 10:36:25.955505 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:25.955459    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-9kr76\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="65a3beb6-121f-4fba-81af-cb5c73bb3b66" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:25.957007 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.956982    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n" event={"ID":"7f109c05-a348-4f2f-9124-c424af9b39de","Type":"ContainerStarted","Data":"a7d628417a6dec23fc1d624be8eeead444698f6f073351078cbf4a56a2700ec7"}
Sep 06 10:36:25.957127 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.957030    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n" event={"ID":"7f109c05-a348-4f2f-9124-c424af9b39de","Type":"ContainerStarted","Data":"5935087463bddb0c1f014cc685283b6d990645b19ebf2b27da4b917c1a524220"}
Sep 06 10:36:25.957127 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.957070    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n"
Sep 06 10:36:25.957127 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:25.957096    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-gmfnm\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="8f390f7c-1af3-4b6a-9212-552cc4bc7aa7" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:25.958648 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.958621    2557 generic.go:356] "Generic (PLEG): container finished" podID="6d80ae13-b32e-4bed-b781-75eaae79a209" containerID="ba0825000e4cb3b6fbf9d0d80206f97f5874ce267279b6fdee22cbd53ecf4230" exitCode=0
Sep 06 10:36:25.958761 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.958659    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"
Sep 06 10:36:25.958761 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.958709    2557 scope.go:117] "RemoveContainer" containerID="ba0825000e4cb3b6fbf9d0d80206f97f5874ce267279b6fdee22cbd53ecf4230"
Sep 06 10:36:25.958869 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:25.958779    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-qqwl9\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="6d80ae13-b32e-4bed-b781-75eaae79a209" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"
Sep 06 10:36:25.960203 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.960183    2557 generic.go:356] "Generic (PLEG): container finished" podID="65a3beb6-121f-4fba-81af-cb5c73bb3b66" containerID="711e5fdcb9bd61bdf4de2d3e3398ae756229c1aef94832eb8dd0fbac8e0eccef" exitCode=0
Sep 06 10:36:25.960283 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.960235    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:25.960831 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:25.960806    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-9kr76\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="65a3beb6-121f-4fba-81af-cb5c73bb3b66" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:25.962661 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:25.962595    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-gmfnm\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="8f390f7c-1af3-4b6a-9212-552cc4bc7aa7" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:25.970639 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.970619    2557 scope.go:117] "RemoveContainer" containerID="ba0825000e4cb3b6fbf9d0d80206f97f5874ce267279b6fdee22cbd53ecf4230"
Sep 06 10:36:25.970909 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:25.970889    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba0825000e4cb3b6fbf9d0d80206f97f5874ce267279b6fdee22cbd53ecf4230\": container with ID starting with ba0825000e4cb3b6fbf9d0d80206f97f5874ce267279b6fdee22cbd53ecf4230 not found: ID does not exist" containerID="ba0825000e4cb3b6fbf9d0d80206f97f5874ce267279b6fdee22cbd53ecf4230"
Sep 06 10:36:25.970986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.970920    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba0825000e4cb3b6fbf9d0d80206f97f5874ce267279b6fdee22cbd53ecf4230"} err="failed to get container status \"ba0825000e4cb3b6fbf9d0d80206f97f5874ce267279b6fdee22cbd53ecf4230\": rpc error: code = NotFound desc = could not find container \"ba0825000e4cb3b6fbf9d0d80206f97f5874ce267279b6fdee22cbd53ecf4230\": container with ID starting with ba0825000e4cb3b6fbf9d0d80206f97f5874ce267279b6fdee22cbd53ecf4230 not found: ID does not exist"
Sep 06 10:36:25.970986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.970942    2557 scope.go:117] "RemoveContainer" containerID="711e5fdcb9bd61bdf4de2d3e3398ae756229c1aef94832eb8dd0fbac8e0eccef"
Sep 06 10:36:25.981812 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.981773    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n" podStartSLOduration=1.981761939 podStartE2EDuration="1.981761939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:36:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:36:25.981114662 +0000 UTC m=+398.430895429" watchObservedRunningTime="2026-09-06 10:36:25.981761939 +0000 UTC m=+398.431542706"
Sep 06 10:36:25.999611 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:25.999563    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jwtd2" podStartSLOduration=1.999550752 podStartE2EDuration="1.999550752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:36:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:36:25.997748974 +0000 UTC m=+398.447529745" watchObservedRunningTime="2026-09-06 10:36:25.999550752 +0000 UTC m=+398.449331520"
Sep 06 10:36:25.999717 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:25.999605    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-qqwl9\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="6d80ae13-b32e-4bed-b781-75eaae79a209" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"
Sep 06 10:36:26.001560 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:26.001538    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-9kr76\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="65a3beb6-121f-4fba-81af-cb5c73bb3b66" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:26.003353 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:26.003327    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-gmfnm\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="8f390f7c-1af3-4b6a-9212-552cc4bc7aa7" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:26.005201 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:26.005179    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-qqwl9\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="6d80ae13-b32e-4bed-b781-75eaae79a209" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"
Sep 06 10:36:26.006895 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:26.006875    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-qqwl9\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="6d80ae13-b32e-4bed-b781-75eaae79a209" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qqwl9"
Sep 06 10:36:26.008506 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:26.008478    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-9kr76\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="65a3beb6-121f-4fba-81af-cb5c73bb3b66" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9kr76"
Sep 06 10:36:26.010195 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:26.010177    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-gmfnm\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="8f390f7c-1af3-4b6a-9212-552cc4bc7aa7" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:26.035995 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:26.035965    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65a3beb6-121f-4fba-81af-cb5c73bb3b66" path="/var/lib/kubelet/pods/65a3beb6-121f-4fba-81af-cb5c73bb3b66/volumes"
Sep 06 10:36:26.036331 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:26.036313    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d80ae13-b32e-4bed-b781-75eaae79a209" path="/var/lib/kubelet/pods/6d80ae13-b32e-4bed-b781-75eaae79a209/volumes"
Sep 06 10:36:26.065343 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:26.065324    2557 scope.go:117] "RemoveContainer" containerID="711e5fdcb9bd61bdf4de2d3e3398ae756229c1aef94832eb8dd0fbac8e0eccef"
Sep 06 10:36:26.065727 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:26.065701    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"711e5fdcb9bd61bdf4de2d3e3398ae756229c1aef94832eb8dd0fbac8e0eccef\": container with ID starting with 711e5fdcb9bd61bdf4de2d3e3398ae756229c1aef94832eb8dd0fbac8e0eccef not found: ID does not exist" containerID="711e5fdcb9bd61bdf4de2d3e3398ae756229c1aef94832eb8dd0fbac8e0eccef"
Sep 06 10:36:26.065854 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:26.065734    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"711e5fdcb9bd61bdf4de2d3e3398ae756229c1aef94832eb8dd0fbac8e0eccef"} err="failed to get container status \"711e5fdcb9bd61bdf4de2d3e3398ae756229c1aef94832eb8dd0fbac8e0eccef\": rpc error: code = NotFound desc = could not find container \"711e5fdcb9bd61bdf4de2d3e3398ae756229c1aef94832eb8dd0fbac8e0eccef\": container with ID starting with 711e5fdcb9bd61bdf4de2d3e3398ae756229c1aef94832eb8dd0fbac8e0eccef not found: ID does not exist"
Sep 06 10:36:26.190355 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:26.190327    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:26.296552 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:26.296463    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wjh6\" (UniqueName: \"kubernetes.io/projected/8f390f7c-1af3-4b6a-9212-552cc4bc7aa7-kube-api-access-5wjh6\") pod \"8f390f7c-1af3-4b6a-9212-552cc4bc7aa7\" (UID: \"8f390f7c-1af3-4b6a-9212-552cc4bc7aa7\") "
Sep 06 10:36:26.296552 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:26.296523    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f390f7c-1af3-4b6a-9212-552cc4bc7aa7-extensions-socket-volume\") pod \"8f390f7c-1af3-4b6a-9212-552cc4bc7aa7\" (UID: \"8f390f7c-1af3-4b6a-9212-552cc4bc7aa7\") "
Sep 06 10:36:26.296858 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:26.296835    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f390f7c-1af3-4b6a-9212-552cc4bc7aa7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8f390f7c-1af3-4b6a-9212-552cc4bc7aa7" (UID: "8f390f7c-1af3-4b6a-9212-552cc4bc7aa7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:36:26.298407 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:26.298385    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f390f7c-1af3-4b6a-9212-552cc4bc7aa7-kube-api-access-5wjh6" (OuterVolumeSpecName: "kube-api-access-5wjh6") pod "8f390f7c-1af3-4b6a-9212-552cc4bc7aa7" (UID: "8f390f7c-1af3-4b6a-9212-552cc4bc7aa7"). InnerVolumeSpecName "kube-api-access-5wjh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:36:26.397636 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:26.397608    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wjh6\" (UniqueName: \"kubernetes.io/projected/8f390f7c-1af3-4b6a-9212-552cc4bc7aa7-kube-api-access-5wjh6\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:36:26.397636 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:26.397632    2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f390f7c-1af3-4b6a-9212-552cc4bc7aa7-extensions-socket-volume\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:36:26.966408 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:26.966373    2557 generic.go:356] "Generic (PLEG): container finished" podID="8f390f7c-1af3-4b6a-9212-552cc4bc7aa7" containerID="e2f7cc73620fb6706467f5075b5e86915e650dfc06a5e87434f96cbf70b6eea9" exitCode=0
Sep 06 10:36:26.966863 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:26.966421    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:26.966863 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:26.966477    2557 scope.go:117] "RemoveContainer" containerID="e2f7cc73620fb6706467f5075b5e86915e650dfc06a5e87434f96cbf70b6eea9"
Sep 06 10:36:26.968728 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:26.968702    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-gmfnm\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="8f390f7c-1af3-4b6a-9212-552cc4bc7aa7" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:26.975545 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:26.975527    2557 scope.go:117] "RemoveContainer" containerID="e2f7cc73620fb6706467f5075b5e86915e650dfc06a5e87434f96cbf70b6eea9"
Sep 06 10:36:26.975805 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:26.975788    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2f7cc73620fb6706467f5075b5e86915e650dfc06a5e87434f96cbf70b6eea9\": container with ID starting with e2f7cc73620fb6706467f5075b5e86915e650dfc06a5e87434f96cbf70b6eea9 not found: ID does not exist" containerID="e2f7cc73620fb6706467f5075b5e86915e650dfc06a5e87434f96cbf70b6eea9"
Sep 06 10:36:26.975851 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:26.975812    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2f7cc73620fb6706467f5075b5e86915e650dfc06a5e87434f96cbf70b6eea9"} err="failed to get container status \"e2f7cc73620fb6706467f5075b5e86915e650dfc06a5e87434f96cbf70b6eea9\": rpc error: code = NotFound desc = could not find container \"e2f7cc73620fb6706467f5075b5e86915e650dfc06a5e87434f96cbf70b6eea9\": container with ID starting with e2f7cc73620fb6706467f5075b5e86915e650dfc06a5e87434f96cbf70b6eea9 not found: ID does not exist"
Sep 06 10:36:26.976967 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:26.976947    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-gmfnm\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="8f390f7c-1af3-4b6a-9212-552cc4bc7aa7" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:28.033915 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:28.033881    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-gmfnm\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="8f390f7c-1af3-4b6a-9212-552cc4bc7aa7" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-gmfnm"
Sep 06 10:36:28.036234 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:28.036209    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f390f7c-1af3-4b6a-9212-552cc4bc7aa7" path="/var/lib/kubelet/pods/8f390f7c-1af3-4b6a-9212-552cc4bc7aa7/volumes"
Sep 06 10:36:36.972917 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:36.972168    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jwtd2"
Sep 06 10:36:36.972917 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:36.972305    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n"
Sep 06 10:36:37.650363 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:37.650299    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp"]
Sep 06 10:36:37.657271 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:37.657250    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp"
Sep 06 10:36:37.669813 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:37.669793    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp"]
Sep 06 10:36:37.783839 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:37.783811    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfd6q\" (UniqueName: \"kubernetes.io/projected/91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1-kube-api-access-dfd6q\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-rj4tp\" (UID: \"91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp"
Sep 06 10:36:37.783977 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:37.783869    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-rj4tp\" (UID: \"91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp"
Sep 06 10:36:37.885120 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:37.885094    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfd6q\" (UniqueName: \"kubernetes.io/projected/91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1-kube-api-access-dfd6q\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-rj4tp\" (UID: \"91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp"
Sep 06 10:36:37.885259 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:37.885161    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-rj4tp\" (UID: \"91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp"
Sep 06 10:36:37.885591 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:37.885574    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-rj4tp\" (UID: \"91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp"
Sep 06 10:36:37.893780 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:37.893759    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfd6q\" (UniqueName: \"kubernetes.io/projected/91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1-kube-api-access-dfd6q\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-rj4tp\" (UID: \"91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp"
Sep 06 10:36:37.967284 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:37.967224    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp"
Sep 06 10:36:38.244562 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:38.244538    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp"]
Sep 06 10:36:38.246587 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:36:38.246557    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91e80eef_0905_4ab9_bf55_bc1ddaf7f8c1.slice/crio-51afe812bea64803ef6c434cebebc3490d8d4796015ecd208ecf89b1b1fea238 WatchSource:0}: Error finding container 51afe812bea64803ef6c434cebebc3490d8d4796015ecd208ecf89b1b1fea238: Status 404 returned error can't find the container with id 51afe812bea64803ef6c434cebebc3490d8d4796015ecd208ecf89b1b1fea238
Sep 06 10:36:39.033621 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:39.033591    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp" event={"ID":"91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1","Type":"ContainerStarted","Data":"c9b6c3fcc66f73208c69fec083966b7e1bcd376fbc640e6c93a4131af5406eee"}
Sep 06 10:36:39.033789 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:39.033639    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp"
Sep 06 10:36:39.033789 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:39.033662    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp" event={"ID":"91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1","Type":"ContainerStarted","Data":"51afe812bea64803ef6c434cebebc3490d8d4796015ecd208ecf89b1b1fea238"}
Sep 06 10:36:39.071313 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:39.071250    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp" podStartSLOduration=2.071231077 podStartE2EDuration="2.071231077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:36:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:36:39.069418566 +0000 UTC m=+411.519199335" watchObservedRunningTime="2026-09-06 10:36:39.071231077 +0000 UTC m=+411.521011849"
Sep 06 10:36:50.041144 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:50.041118    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp"
Sep 06 10:36:50.090064 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:50.090022    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n"]
Sep 06 10:36:50.090277 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:50.090246    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n" podUID="7f109c05-a348-4f2f-9124-c424af9b39de" containerName="manager" containerID="cri-o://a7d628417a6dec23fc1d624be8eeead444698f6f073351078cbf4a56a2700ec7" gracePeriod=10
Sep 06 10:36:50.330285 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:50.330264    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n"
Sep 06 10:36:50.481963 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:50.481929    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gp6p\" (UniqueName: \"kubernetes.io/projected/7f109c05-a348-4f2f-9124-c424af9b39de-kube-api-access-4gp6p\") pod \"7f109c05-a348-4f2f-9124-c424af9b39de\" (UID: \"7f109c05-a348-4f2f-9124-c424af9b39de\") "
Sep 06 10:36:50.482121 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:50.481995    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f109c05-a348-4f2f-9124-c424af9b39de-extensions-socket-volume\") pod \"7f109c05-a348-4f2f-9124-c424af9b39de\" (UID: \"7f109c05-a348-4f2f-9124-c424af9b39de\") "
Sep 06 10:36:50.482451 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:50.482427    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f109c05-a348-4f2f-9124-c424af9b39de-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7f109c05-a348-4f2f-9124-c424af9b39de" (UID: "7f109c05-a348-4f2f-9124-c424af9b39de"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:36:50.484093 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:50.484070    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f109c05-a348-4f2f-9124-c424af9b39de-kube-api-access-4gp6p" (OuterVolumeSpecName: "kube-api-access-4gp6p") pod "7f109c05-a348-4f2f-9124-c424af9b39de" (UID: "7f109c05-a348-4f2f-9124-c424af9b39de"). InnerVolumeSpecName "kube-api-access-4gp6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:36:50.583627 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:50.583562    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gp6p\" (UniqueName: \"kubernetes.io/projected/7f109c05-a348-4f2f-9124-c424af9b39de-kube-api-access-4gp6p\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:36:50.583627 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:50.583592    2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f109c05-a348-4f2f-9124-c424af9b39de-extensions-socket-volume\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:36:51.090559 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:51.090521    2557 generic.go:356] "Generic (PLEG): container finished" podID="7f109c05-a348-4f2f-9124-c424af9b39de" containerID="a7d628417a6dec23fc1d624be8eeead444698f6f073351078cbf4a56a2700ec7" exitCode=0
Sep 06 10:36:51.091021 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:51.090602    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n"
Sep 06 10:36:51.091021 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:51.090606    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n" event={"ID":"7f109c05-a348-4f2f-9124-c424af9b39de","Type":"ContainerDied","Data":"a7d628417a6dec23fc1d624be8eeead444698f6f073351078cbf4a56a2700ec7"}
Sep 06 10:36:51.091021 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:51.090656    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n" event={"ID":"7f109c05-a348-4f2f-9124-c424af9b39de","Type":"ContainerDied","Data":"5935087463bddb0c1f014cc685283b6d990645b19ebf2b27da4b917c1a524220"}
Sep 06 10:36:51.091021 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:51.090678    2557 scope.go:117] "RemoveContainer" containerID="a7d628417a6dec23fc1d624be8eeead444698f6f073351078cbf4a56a2700ec7"
Sep 06 10:36:51.098956 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:51.098940    2557 scope.go:117] "RemoveContainer" containerID="a7d628417a6dec23fc1d624be8eeead444698f6f073351078cbf4a56a2700ec7"
Sep 06 10:36:51.099209 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:36:51.099190    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7d628417a6dec23fc1d624be8eeead444698f6f073351078cbf4a56a2700ec7\": container with ID starting with a7d628417a6dec23fc1d624be8eeead444698f6f073351078cbf4a56a2700ec7 not found: ID does not exist" containerID="a7d628417a6dec23fc1d624be8eeead444698f6f073351078cbf4a56a2700ec7"
Sep 06 10:36:51.099271 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:51.099217    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7d628417a6dec23fc1d624be8eeead444698f6f073351078cbf4a56a2700ec7"} err="failed to get container status \"a7d628417a6dec23fc1d624be8eeead444698f6f073351078cbf4a56a2700ec7\": rpc error: code = NotFound desc = could not find container \"a7d628417a6dec23fc1d624be8eeead444698f6f073351078cbf4a56a2700ec7\": container with ID starting with a7d628417a6dec23fc1d624be8eeead444698f6f073351078cbf4a56a2700ec7 not found: ID does not exist"
Sep 06 10:36:51.112025 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:51.112005    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n"]
Sep 06 10:36:51.116063 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:51.116043    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-bzm9n"]
Sep 06 10:36:52.036341 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:52.036305    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f109c05-a348-4f2f-9124-c424af9b39de" path="/var/lib/kubelet/pods/7f109c05-a348-4f2f-9124-c424af9b39de/volumes"
Sep 06 10:36:53.420132 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.420104    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"]
Sep 06 10:36:53.420507 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.420470    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f109c05-a348-4f2f-9124-c424af9b39de" containerName="manager"
Sep 06 10:36:53.420507 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.420481    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f109c05-a348-4f2f-9124-c424af9b39de" containerName="manager"
Sep 06 10:36:53.420592 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.420563    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f109c05-a348-4f2f-9124-c424af9b39de" containerName="manager"
Sep 06 10:36:53.423601 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.423584    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.425756 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.425734    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 06 10:36:53.425886 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.425846    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-h87fw\""
Sep 06 10:36:53.435298 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.435271    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"]
Sep 06 10:36:53.507739 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.507712    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e05e829-48be-442d-8e9b-715b4604dc63-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.507848 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.507760    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e05e829-48be-442d-8e9b-715b4604dc63-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.507848 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.507798    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rbft\" (UniqueName: \"kubernetes.io/projected/8e05e829-48be-442d-8e9b-715b4604dc63-kube-api-access-8rbft\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.507848 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.507836    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e05e829-48be-442d-8e9b-715b4604dc63-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.507950 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.507905    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e05e829-48be-442d-8e9b-715b4604dc63-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.507950 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.507937    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e05e829-48be-442d-8e9b-715b4604dc63-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.508014 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.507957    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e05e829-48be-442d-8e9b-715b4604dc63-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.508046 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.508017    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e05e829-48be-442d-8e9b-715b4604dc63-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.508080 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.508045    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e05e829-48be-442d-8e9b-715b4604dc63-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.609026 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.608997    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e05e829-48be-442d-8e9b-715b4604dc63-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.609026 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.609029    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e05e829-48be-442d-8e9b-715b4604dc63-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.609212 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.609053    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e05e829-48be-442d-8e9b-715b4604dc63-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.609212 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.609076    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e05e829-48be-442d-8e9b-715b4604dc63-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.609212 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.609102    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e05e829-48be-442d-8e9b-715b4604dc63-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.609212 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.609159    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e05e829-48be-442d-8e9b-715b4604dc63-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.609212 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.609191    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e05e829-48be-442d-8e9b-715b4604dc63-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.609471 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.609222    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rbft\" (UniqueName: \"kubernetes.io/projected/8e05e829-48be-442d-8e9b-715b4604dc63-kube-api-access-8rbft\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.609471 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.609254    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e05e829-48be-442d-8e9b-715b4604dc63-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.609904 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.609879    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e05e829-48be-442d-8e9b-715b4604dc63-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.610087 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.610064    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e05e829-48be-442d-8e9b-715b4604dc63-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.610162 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.610134    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e05e829-48be-442d-8e9b-715b4604dc63-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.610235 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.610146    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e05e829-48be-442d-8e9b-715b4604dc63-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.610235 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.610185    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e05e829-48be-442d-8e9b-715b4604dc63-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.611777 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.611749    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e05e829-48be-442d-8e9b-715b4604dc63-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.611874 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.611829    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e05e829-48be-442d-8e9b-715b4604dc63-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.616926 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.616900    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rbft\" (UniqueName: \"kubernetes.io/projected/8e05e829-48be-442d-8e9b-715b4604dc63-kube-api-access-8rbft\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.616926 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.616913    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e05e829-48be-442d-8e9b-715b4604dc63-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cqglj\" (UID: \"8e05e829-48be-442d-8e9b-715b4604dc63\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:53.734037 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:53.733986    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:54.006454 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:54.006421    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"]
Sep 06 10:36:54.008912 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:36:54.008877    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e05e829_48be_442d_8e9b_715b4604dc63.slice/crio-23c0601ec6dc58819720618724e719129ec4f3fcc991bb0e2ebb1d846b92de01 WatchSource:0}: Error finding container 23c0601ec6dc58819720618724e719129ec4f3fcc991bb0e2ebb1d846b92de01: Status 404 returned error can't find the container with id 23c0601ec6dc58819720618724e719129ec4f3fcc991bb0e2ebb1d846b92de01
Sep 06 10:36:54.106176 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:54.106144    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj" event={"ID":"8e05e829-48be-442d-8e9b-715b4604dc63","Type":"ContainerStarted","Data":"23c0601ec6dc58819720618724e719129ec4f3fcc991bb0e2ebb1d846b92de01"}
Sep 06 10:36:56.486300 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:56.486259    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:36:56.486592 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:56.486341    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:36:56.486592 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:56.486387    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:36:57.123199 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:57.123162    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj" event={"ID":"8e05e829-48be-442d-8e9b-715b4604dc63","Type":"ContainerStarted","Data":"237d9e3331b50f9698bc0ff592640597682d8649d2c5119e86414d0b9986973e"}
Sep 06 10:36:57.142989 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:57.142940    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj" podStartSLOduration=1.667990764 podStartE2EDuration="4.142926981s" totalImagesPullingTime="2.474936217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:36:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:36:57.141771933 +0000 UTC m=+429.591552702" watchObservedRunningTime="2026-09-06 10:36:57.142926981 +0000 UTC m=+429.592707750"
Sep 06 10:36:57.734313 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:57.734272    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:57.739203 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:57.739179    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:58.127460 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:58.127434    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:36:58.128526 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:36:58.128505    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:37:07.374712 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:07.374669    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:37:07.379387 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:07.379364    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"
Sep 06 10:37:07.381979 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:07.381957    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 06 10:37:07.382076 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:07.381959    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-t248c\""
Sep 06 10:37:07.386687 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:07.386664    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:37:07.414328 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:07.414285    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:37:07.424842 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:07.424815    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw6tj\" (UniqueName: \"kubernetes.io/projected/4a59eacd-9266-4b63-a38d-8363c1378636-kube-api-access-dw6tj\") pod \"limitador-limitador-85d9db4d5d-k6wsv\" (UID: \"4a59eacd-9266-4b63-a38d-8363c1378636\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"
Sep 06 10:37:07.424953 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:07.424877    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4a59eacd-9266-4b63-a38d-8363c1378636-config-file\") pod \"limitador-limitador-85d9db4d5d-k6wsv\" (UID: \"4a59eacd-9266-4b63-a38d-8363c1378636\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"
Sep 06 10:37:07.525801 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:07.525770    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw6tj\" (UniqueName: \"kubernetes.io/projected/4a59eacd-9266-4b63-a38d-8363c1378636-kube-api-access-dw6tj\") pod \"limitador-limitador-85d9db4d5d-k6wsv\" (UID: \"4a59eacd-9266-4b63-a38d-8363c1378636\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"
Sep 06 10:37:07.525966 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:07.525855    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4a59eacd-9266-4b63-a38d-8363c1378636-config-file\") pod \"limitador-limitador-85d9db4d5d-k6wsv\" (UID: \"4a59eacd-9266-4b63-a38d-8363c1378636\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"
Sep 06 10:37:07.526515 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:07.526482    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4a59eacd-9266-4b63-a38d-8363c1378636-config-file\") pod \"limitador-limitador-85d9db4d5d-k6wsv\" (UID: \"4a59eacd-9266-4b63-a38d-8363c1378636\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"
Sep 06 10:37:07.534187 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:07.534162    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw6tj\" (UniqueName: \"kubernetes.io/projected/4a59eacd-9266-4b63-a38d-8363c1378636-kube-api-access-dw6tj\") pod \"limitador-limitador-85d9db4d5d-k6wsv\" (UID: \"4a59eacd-9266-4b63-a38d-8363c1378636\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"
Sep 06 10:37:07.689962 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:07.689880    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"
Sep 06 10:37:07.834754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:07.834720    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-pwndn"]
Sep 06 10:37:07.839406 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:07.839384    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-pwndn"
Sep 06 10:37:07.841732 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:07.841709    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mjjjs\""
Sep 06 10:37:07.845155 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:07.845133    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-pwndn"]
Sep 06 10:37:07.929424 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:07.929387    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcst9\" (UniqueName: \"kubernetes.io/projected/4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9-kube-api-access-vcst9\") pod \"authorino-c548b5d7d-pwndn\" (UID: \"4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9\") " pod="kuadrant-system/authorino-c548b5d7d-pwndn"
Sep 06 10:37:07.962118 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:07.962090    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:37:07.964743 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:37:07.964715    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a59eacd_9266_4b63_a38d_8363c1378636.slice/crio-7b80085414d387973f2c630a02cd94cdb2a87ad76b45ec41d943513a6c4daa9c WatchSource:0}: Error finding container 7b80085414d387973f2c630a02cd94cdb2a87ad76b45ec41d943513a6c4daa9c: Status 404 returned error can't find the container with id 7b80085414d387973f2c630a02cd94cdb2a87ad76b45ec41d943513a6c4daa9c
Sep 06 10:37:08.030139 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:08.030107    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcst9\" (UniqueName: \"kubernetes.io/projected/4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9-kube-api-access-vcst9\") pod \"authorino-c548b5d7d-pwndn\" (UID: \"4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9\") " pod="kuadrant-system/authorino-c548b5d7d-pwndn"
Sep 06 10:37:08.038539 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:08.038483    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcst9\" (UniqueName: \"kubernetes.io/projected/4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9-kube-api-access-vcst9\") pod \"authorino-c548b5d7d-pwndn\" (UID: \"4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9\") " pod="kuadrant-system/authorino-c548b5d7d-pwndn"
Sep 06 10:37:08.150641 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:08.150615    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-pwndn"
Sep 06 10:37:08.177396 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:08.177368    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv" event={"ID":"4a59eacd-9266-4b63-a38d-8363c1378636","Type":"ContainerStarted","Data":"7b80085414d387973f2c630a02cd94cdb2a87ad76b45ec41d943513a6c4daa9c"}
Sep 06 10:37:08.419227 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:08.419200    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-pwndn"]
Sep 06 10:37:08.421706 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:37:08.421679    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4527fd1e_b9aa_43cc_9913_fdad4c8ce4b9.slice/crio-57d55d6edeb067c4f0e68433944e9ea0906d8f3421046f79ea671281b6030982 WatchSource:0}: Error finding container 57d55d6edeb067c4f0e68433944e9ea0906d8f3421046f79ea671281b6030982: Status 404 returned error can't find the container with id 57d55d6edeb067c4f0e68433944e9ea0906d8f3421046f79ea671281b6030982
Sep 06 10:37:09.185527 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:09.185471    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv" event={"ID":"4a59eacd-9266-4b63-a38d-8363c1378636","Type":"ContainerStarted","Data":"43ed85633e6155cabdbba7a2b1f1f931f312749ce5db628fe06ef5f8c1de7b3a"}
Sep 06 10:37:09.185692 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:09.185567    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"
Sep 06 10:37:09.187209 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:09.187182    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-pwndn" event={"ID":"4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9","Type":"ContainerStarted","Data":"57d55d6edeb067c4f0e68433944e9ea0906d8f3421046f79ea671281b6030982"}
Sep 06 10:37:09.207104 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:09.207053    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv" podStartSLOduration=1.140676238 podStartE2EDuration="2.207034109s" totalImagesPullingTime="1.066357871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:37:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:37:09.205980353 +0000 UTC m=+441.655761120" watchObservedRunningTime="2026-09-06 10:37:09.207034109 +0000 UTC m=+441.656814878"
Sep 06 10:37:11.199508 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:11.199450    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-pwndn" event={"ID":"4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9","Type":"ContainerStarted","Data":"5f434c2f2306e1952acdd7d365e9a598e51dcf20594a6de945a6179f2ad5bd62"}
Sep 06 10:37:11.215108 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:11.215059    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-pwndn" podStartSLOduration=2.424178957 podStartE2EDuration="4.215043917s" totalImagesPullingTime="1.79086496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:37:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:37:11.213889828 +0000 UTC m=+443.663670597" watchObservedRunningTime="2026-09-06 10:37:11.215043917 +0000 UTC m=+443.664824684"
Sep 06 10:37:20.193808 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:20.193779    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"
Sep 06 10:37:37.115634 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:37.115602    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7c6f99d97b-z22xw"]
Sep 06 10:37:37.120575 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:37.120555    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c6f99d97b-z22xw"
Sep 06 10:37:37.123147 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:37.123115    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 06 10:37:37.124976 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:37.124955    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c6f99d97b-z22xw"]
Sep 06 10:37:37.262598 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:37.262567    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7xvb\" (UniqueName: \"kubernetes.io/projected/f34ebafc-298f-47d3-9af1-5611da648107-kube-api-access-t7xvb\") pod \"authorino-7c6f99d97b-z22xw\" (UID: \"f34ebafc-298f-47d3-9af1-5611da648107\") " pod="kuadrant-system/authorino-7c6f99d97b-z22xw"
Sep 06 10:37:37.262738 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:37.262623    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f34ebafc-298f-47d3-9af1-5611da648107-tls-cert\") pod \"authorino-7c6f99d97b-z22xw\" (UID: \"f34ebafc-298f-47d3-9af1-5611da648107\") " pod="kuadrant-system/authorino-7c6f99d97b-z22xw"
Sep 06 10:37:37.363860 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:37.363829    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7xvb\" (UniqueName: \"kubernetes.io/projected/f34ebafc-298f-47d3-9af1-5611da648107-kube-api-access-t7xvb\") pod \"authorino-7c6f99d97b-z22xw\" (UID: \"f34ebafc-298f-47d3-9af1-5611da648107\") " pod="kuadrant-system/authorino-7c6f99d97b-z22xw"
Sep 06 10:37:37.364021 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:37.363879    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f34ebafc-298f-47d3-9af1-5611da648107-tls-cert\") pod \"authorino-7c6f99d97b-z22xw\" (UID: \"f34ebafc-298f-47d3-9af1-5611da648107\") " pod="kuadrant-system/authorino-7c6f99d97b-z22xw"
Sep 06 10:37:37.366279 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:37.366218    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f34ebafc-298f-47d3-9af1-5611da648107-tls-cert\") pod \"authorino-7c6f99d97b-z22xw\" (UID: \"f34ebafc-298f-47d3-9af1-5611da648107\") " pod="kuadrant-system/authorino-7c6f99d97b-z22xw"
Sep 06 10:37:37.371238 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:37.371209    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7xvb\" (UniqueName: \"kubernetes.io/projected/f34ebafc-298f-47d3-9af1-5611da648107-kube-api-access-t7xvb\") pod \"authorino-7c6f99d97b-z22xw\" (UID: \"f34ebafc-298f-47d3-9af1-5611da648107\") " pod="kuadrant-system/authorino-7c6f99d97b-z22xw"
Sep 06 10:37:37.430020 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:37.429978    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c6f99d97b-z22xw"
Sep 06 10:37:37.696605 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:37.696529    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c6f99d97b-z22xw"]
Sep 06 10:37:37.700116 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:37:37.700090    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf34ebafc_298f_47d3_9af1_5611da648107.slice/crio-c050a82508b0f88dd052a8cefdaa6f161b8db38e1fb52fa59bbb057c739b9502 WatchSource:0}: Error finding container c050a82508b0f88dd052a8cefdaa6f161b8db38e1fb52fa59bbb057c739b9502: Status 404 returned error can't find the container with id c050a82508b0f88dd052a8cefdaa6f161b8db38e1fb52fa59bbb057c739b9502
Sep 06 10:37:38.324241 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:38.324200    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c6f99d97b-z22xw" event={"ID":"f34ebafc-298f-47d3-9af1-5611da648107","Type":"ContainerStarted","Data":"c050a82508b0f88dd052a8cefdaa6f161b8db38e1fb52fa59bbb057c739b9502"}
Sep 06 10:37:39.330129 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:39.330095    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c6f99d97b-z22xw" event={"ID":"f34ebafc-298f-47d3-9af1-5611da648107","Type":"ContainerStarted","Data":"96f7d34924971acdcc739acc2ead63ce3d3a59545cd7c858a243d5c959b3c931"}
Sep 06 10:37:39.346390 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:39.346336    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7c6f99d97b-z22xw" podStartSLOduration=1.481521314 podStartE2EDuration="2.34631823s" totalImagesPullingTime="864.796916ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:37:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:37:39.344590255 +0000 UTC m=+471.794371024" watchObservedRunningTime="2026-09-06 10:37:39.34631823 +0000 UTC m=+471.796098995"
Sep 06 10:37:39.369823 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:39.369793    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-pwndn"]
Sep 06 10:37:39.370013 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:39.369988    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-pwndn" podUID="4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9" containerName="authorino" containerID="cri-o://5f434c2f2306e1952acdd7d365e9a598e51dcf20594a6de945a6179f2ad5bd62" gracePeriod=30
Sep 06 10:37:39.608900 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:39.608876    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-pwndn"
Sep 06 10:37:39.688032 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:39.688002    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcst9\" (UniqueName: \"kubernetes.io/projected/4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9-kube-api-access-vcst9\") pod \"4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9\" (UID: \"4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9\") "
Sep 06 10:37:39.690005 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:39.689977    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9-kube-api-access-vcst9" (OuterVolumeSpecName: "kube-api-access-vcst9") pod "4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9" (UID: "4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9"). InnerVolumeSpecName "kube-api-access-vcst9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:37:39.789238 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:39.789198    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcst9\" (UniqueName: \"kubernetes.io/projected/4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9-kube-api-access-vcst9\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:37:40.336009 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:40.335929    2557 generic.go:356] "Generic (PLEG): container finished" podID="4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9" containerID="5f434c2f2306e1952acdd7d365e9a598e51dcf20594a6de945a6179f2ad5bd62" exitCode=0
Sep 06 10:37:40.336009 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:40.335969    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-pwndn"
Sep 06 10:37:40.336465 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:40.336007    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-pwndn" event={"ID":"4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9","Type":"ContainerDied","Data":"5f434c2f2306e1952acdd7d365e9a598e51dcf20594a6de945a6179f2ad5bd62"}
Sep 06 10:37:40.336465 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:40.336043    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-pwndn" event={"ID":"4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9","Type":"ContainerDied","Data":"57d55d6edeb067c4f0e68433944e9ea0906d8f3421046f79ea671281b6030982"}
Sep 06 10:37:40.336465 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:40.336060    2557 scope.go:117] "RemoveContainer" containerID="5f434c2f2306e1952acdd7d365e9a598e51dcf20594a6de945a6179f2ad5bd62"
Sep 06 10:37:40.344526 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:40.344508    2557 scope.go:117] "RemoveContainer" containerID="5f434c2f2306e1952acdd7d365e9a598e51dcf20594a6de945a6179f2ad5bd62"
Sep 06 10:37:40.344817 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:37:40.344797    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f434c2f2306e1952acdd7d365e9a598e51dcf20594a6de945a6179f2ad5bd62\": container with ID starting with 5f434c2f2306e1952acdd7d365e9a598e51dcf20594a6de945a6179f2ad5bd62 not found: ID does not exist" containerID="5f434c2f2306e1952acdd7d365e9a598e51dcf20594a6de945a6179f2ad5bd62"
Sep 06 10:37:40.344897 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:40.344824    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f434c2f2306e1952acdd7d365e9a598e51dcf20594a6de945a6179f2ad5bd62"} err="failed to get container status \"5f434c2f2306e1952acdd7d365e9a598e51dcf20594a6de945a6179f2ad5bd62\": rpc error: code = NotFound desc = could not find container \"5f434c2f2306e1952acdd7d365e9a598e51dcf20594a6de945a6179f2ad5bd62\": container with ID starting with 5f434c2f2306e1952acdd7d365e9a598e51dcf20594a6de945a6179f2ad5bd62 not found: ID does not exist"
Sep 06 10:37:40.352297 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:40.352271    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-pwndn"]
Sep 06 10:37:40.355473 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:40.355454    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-pwndn"]
Sep 06 10:37:42.036525 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:37:42.036473    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9" path="/var/lib/kubelet/pods/4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9/volumes"
Sep 06 10:38:22.177856 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:22.177817    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:38:31.590318 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:31.590274    2557 generic.go:356] "Generic (PLEG): container finished" podID="8e05e829-48be-442d-8e9b-715b4604dc63" containerID="237d9e3331b50f9698bc0ff592640597682d8649d2c5119e86414d0b9986973e" exitCode=0
Sep 06 10:38:31.590724 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:31.590343    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj" event={"ID":"8e05e829-48be-442d-8e9b-715b4604dc63","Type":"ContainerDied","Data":"237d9e3331b50f9698bc0ff592640597682d8649d2c5119e86414d0b9986973e"}
Sep 06 10:38:31.590724 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:31.590720    2557 scope.go:117] "RemoveContainer" containerID="237d9e3331b50f9698bc0ff592640597682d8649d2c5119e86414d0b9986973e"
Sep 06 10:38:31.591673 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:31.591642    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:38:31.591774 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:31.591726    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:38:31.591831 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:31.591775    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:38:32.597857 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:32.597822    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj" event={"ID":"8e05e829-48be-442d-8e9b-715b4604dc63","Type":"ContainerStarted","Data":"eecbfb0cfeec0079ab38fb1eeda0cc07d5993468506453c8c9e950dcd175443c"}
Sep 06 10:38:32.735093 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:32.735057    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:38:32.735323 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:32.735301    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-cqglj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 06 10:38:32.735377 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:32.735361    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj" podUID="8e05e829-48be-442d-8e9b-715b4604dc63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 06 10:38:33.735121 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:33.735089    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-cqglj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 06 10:38:33.735464 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:33.735145    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj" podUID="8e05e829-48be-442d-8e9b-715b4604dc63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 06 10:38:34.735205 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:34.735169    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-cqglj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 06 10:38:34.735664 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:34.735222    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj" podUID="8e05e829-48be-442d-8e9b-715b4604dc63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 06 10:38:35.734340 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:35.734300    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-cqglj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 06 10:38:35.734508 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:35.734363    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj" podUID="8e05e829-48be-442d-8e9b-715b4604dc63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 06 10:38:36.734694 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:36.734657    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-cqglj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 06 10:38:36.735066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:36.734714    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj" podUID="8e05e829-48be-442d-8e9b-715b4604dc63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 06 10:38:37.734529 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:37.734470    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-cqglj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 06 10:38:37.734704 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:37.734550    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj" podUID="8e05e829-48be-442d-8e9b-715b4604dc63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 06 10:38:38.632877 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:38.632842    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:38:38.738398 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:38.738368    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:38:38.738725 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:38.738701    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:38:38.739476 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:38.739454    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cqglj"
Sep 06 10:38:48.422390 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:48.422302    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:38:53.807144 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.807112    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"]
Sep 06 10:38:53.807574 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.807461    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9" containerName="authorino"
Sep 06 10:38:53.807574 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.807472    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9" containerName="authorino"
Sep 06 10:38:53.807574 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.807572    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4527fd1e-b9aa-43cc-9913-fdad4c8ce4b9" containerName="authorino"
Sep 06 10:38:53.809781 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.809762    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:53.812233 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.812208    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 06 10:38:53.813181 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.813141    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-h68bt\""
Sep 06 10:38:53.813289 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.813203    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 06 10:38:53.813289 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.813204    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 06 10:38:53.820448 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.820428    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"]
Sep 06 10:38:53.879385 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.879360    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df2c0194-ef33-407c-aea4-7ac2671506bc-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp\" (UID: \"df2c0194-ef33-407c-aea4-7ac2671506bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:53.879525 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.879390    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df2c0194-ef33-407c-aea4-7ac2671506bc-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp\" (UID: \"df2c0194-ef33-407c-aea4-7ac2671506bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:53.879525 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.879407    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df2c0194-ef33-407c-aea4-7ac2671506bc-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp\" (UID: \"df2c0194-ef33-407c-aea4-7ac2671506bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:53.879525 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.879425    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df2c0194-ef33-407c-aea4-7ac2671506bc-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp\" (UID: \"df2c0194-ef33-407c-aea4-7ac2671506bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:53.879525 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.879484    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df2c0194-ef33-407c-aea4-7ac2671506bc-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp\" (UID: \"df2c0194-ef33-407c-aea4-7ac2671506bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:53.879671 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.879551    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp7pr\" (UniqueName: \"kubernetes.io/projected/df2c0194-ef33-407c-aea4-7ac2671506bc-kube-api-access-kp7pr\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp\" (UID: \"df2c0194-ef33-407c-aea4-7ac2671506bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:53.980405 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.980377    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df2c0194-ef33-407c-aea4-7ac2671506bc-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp\" (UID: \"df2c0194-ef33-407c-aea4-7ac2671506bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:53.980548 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.980408    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df2c0194-ef33-407c-aea4-7ac2671506bc-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp\" (UID: \"df2c0194-ef33-407c-aea4-7ac2671506bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:53.980548 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.980434    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df2c0194-ef33-407c-aea4-7ac2671506bc-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp\" (UID: \"df2c0194-ef33-407c-aea4-7ac2671506bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:53.980548 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.980459    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df2c0194-ef33-407c-aea4-7ac2671506bc-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp\" (UID: \"df2c0194-ef33-407c-aea4-7ac2671506bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:53.980548 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.980521    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp7pr\" (UniqueName: \"kubernetes.io/projected/df2c0194-ef33-407c-aea4-7ac2671506bc-kube-api-access-kp7pr\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp\" (UID: \"df2c0194-ef33-407c-aea4-7ac2671506bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:53.980762 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.980636    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df2c0194-ef33-407c-aea4-7ac2671506bc-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp\" (UID: \"df2c0194-ef33-407c-aea4-7ac2671506bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:53.980938 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.980913    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df2c0194-ef33-407c-aea4-7ac2671506bc-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp\" (UID: \"df2c0194-ef33-407c-aea4-7ac2671506bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:53.981011 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.980988    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df2c0194-ef33-407c-aea4-7ac2671506bc-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp\" (UID: \"df2c0194-ef33-407c-aea4-7ac2671506bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:53.981191 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.981170    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df2c0194-ef33-407c-aea4-7ac2671506bc-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp\" (UID: \"df2c0194-ef33-407c-aea4-7ac2671506bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:53.982982 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.982962    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df2c0194-ef33-407c-aea4-7ac2671506bc-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp\" (UID: \"df2c0194-ef33-407c-aea4-7ac2671506bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:53.983127 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.983105    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df2c0194-ef33-407c-aea4-7ac2671506bc-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp\" (UID: \"df2c0194-ef33-407c-aea4-7ac2671506bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:53.988735 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:53.988717    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp7pr\" (UniqueName: \"kubernetes.io/projected/df2c0194-ef33-407c-aea4-7ac2671506bc-kube-api-access-kp7pr\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp\" (UID: \"df2c0194-ef33-407c-aea4-7ac2671506bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:54.120954 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:54.120928    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:38:54.390940 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:54.390918    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"]
Sep 06 10:38:54.392603 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:38:54.392576    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf2c0194_ef33_407c_aea4_7ac2671506bc.slice/crio-54defd68a22721c541161263a080f2e5d0ce8bada01387bc25c156650bfcb955 WatchSource:0}: Error finding container 54defd68a22721c541161263a080f2e5d0ce8bada01387bc25c156650bfcb955: Status 404 returned error can't find the container with id 54defd68a22721c541161263a080f2e5d0ce8bada01387bc25c156650bfcb955
Sep 06 10:38:54.704291 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:54.704210    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp" event={"ID":"df2c0194-ef33-407c-aea4-7ac2671506bc","Type":"ContainerStarted","Data":"54defd68a22721c541161263a080f2e5d0ce8bada01387bc25c156650bfcb955"}
Sep 06 10:38:54.826614 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:54.826574    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:38:55.721002 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:55.720909    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp" event={"ID":"df2c0194-ef33-407c-aea4-7ac2671506bc","Type":"ContainerStarted","Data":"93297b024684291f768a2d06ee440435ceb1feafc48153ac39d591aca7dbeeef"}
Sep 06 10:38:55.738140 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:38:55.738090    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp" podStartSLOduration=1.776592441 podStartE2EDuration="2.738075858s" totalImagesPullingTime="961.483417ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:38:55.736381322 +0000 UTC m=+548.186162095" watchObservedRunningTime="2026-09-06 10:38:55.738075858 +0000 UTC m=+548.187856626"
Sep 06 10:39:04.121723 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:04.121690    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:39:04.122185 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:04.121750    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:39:04.134043 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:04.134015    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:39:04.779934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:04.779906    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp"
Sep 06 10:39:16.419359 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:16.419324    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:39:23.216628 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:23.216594    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:39:28.726005 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.725974    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"]
Sep 06 10:39:28.730575 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.730560    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:28.733051 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.733030    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 06 10:39:28.737094 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.737072    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"]
Sep 06 10:39:28.756099 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.756076    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b76b8b8-f168-4f89-8da8-79b479134957-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dx5fd\" (UID: \"9b76b8b8-f168-4f89-8da8-79b479134957\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:28.756201 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.756119    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b76b8b8-f168-4f89-8da8-79b479134957-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dx5fd\" (UID: \"9b76b8b8-f168-4f89-8da8-79b479134957\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:28.756201 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.756139    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b76b8b8-f168-4f89-8da8-79b479134957-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dx5fd\" (UID: \"9b76b8b8-f168-4f89-8da8-79b479134957\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:28.756201 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.756171    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jk2q\" (UniqueName: \"kubernetes.io/projected/9b76b8b8-f168-4f89-8da8-79b479134957-kube-api-access-9jk2q\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dx5fd\" (UID: \"9b76b8b8-f168-4f89-8da8-79b479134957\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:28.756201 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.756197    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b76b8b8-f168-4f89-8da8-79b479134957-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dx5fd\" (UID: \"9b76b8b8-f168-4f89-8da8-79b479134957\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:28.756332 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.756214    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b76b8b8-f168-4f89-8da8-79b479134957-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dx5fd\" (UID: \"9b76b8b8-f168-4f89-8da8-79b479134957\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:28.857338 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.857316    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b76b8b8-f168-4f89-8da8-79b479134957-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dx5fd\" (UID: \"9b76b8b8-f168-4f89-8da8-79b479134957\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:28.857445 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.857356    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b76b8b8-f168-4f89-8da8-79b479134957-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dx5fd\" (UID: \"9b76b8b8-f168-4f89-8da8-79b479134957\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:28.857445 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.857377    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b76b8b8-f168-4f89-8da8-79b479134957-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dx5fd\" (UID: \"9b76b8b8-f168-4f89-8da8-79b479134957\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:28.857573 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.857532    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jk2q\" (UniqueName: \"kubernetes.io/projected/9b76b8b8-f168-4f89-8da8-79b479134957-kube-api-access-9jk2q\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dx5fd\" (UID: \"9b76b8b8-f168-4f89-8da8-79b479134957\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:28.857619 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.857594    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b76b8b8-f168-4f89-8da8-79b479134957-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dx5fd\" (UID: \"9b76b8b8-f168-4f89-8da8-79b479134957\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:28.857619 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.857612    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b76b8b8-f168-4f89-8da8-79b479134957-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dx5fd\" (UID: \"9b76b8b8-f168-4f89-8da8-79b479134957\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:28.857770 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.857738    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b76b8b8-f168-4f89-8da8-79b479134957-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dx5fd\" (UID: \"9b76b8b8-f168-4f89-8da8-79b479134957\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:28.857885 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.857784    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b76b8b8-f168-4f89-8da8-79b479134957-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dx5fd\" (UID: \"9b76b8b8-f168-4f89-8da8-79b479134957\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:28.857956 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.857943    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b76b8b8-f168-4f89-8da8-79b479134957-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dx5fd\" (UID: \"9b76b8b8-f168-4f89-8da8-79b479134957\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:28.862916 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.859606    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b76b8b8-f168-4f89-8da8-79b479134957-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dx5fd\" (UID: \"9b76b8b8-f168-4f89-8da8-79b479134957\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:28.862916 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.860268    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b76b8b8-f168-4f89-8da8-79b479134957-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dx5fd\" (UID: \"9b76b8b8-f168-4f89-8da8-79b479134957\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:28.868749 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:28.868728    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jk2q\" (UniqueName: \"kubernetes.io/projected/9b76b8b8-f168-4f89-8da8-79b479134957-kube-api-access-9jk2q\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dx5fd\" (UID: \"9b76b8b8-f168-4f89-8da8-79b479134957\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:29.041711 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:29.041636    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:29.317114 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:29.317089    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"]
Sep 06 10:39:29.319002 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:39:29.318971    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b76b8b8_f168_4f89_8da8_79b479134957.slice/crio-f7d57679b75e6b123bd2f3806a3747eb2fd12512f87112ebdc5d7ba0de8eb7a1 WatchSource:0}: Error finding container f7d57679b75e6b123bd2f3806a3747eb2fd12512f87112ebdc5d7ba0de8eb7a1: Status 404 returned error can't find the container with id f7d57679b75e6b123bd2f3806a3747eb2fd12512f87112ebdc5d7ba0de8eb7a1
Sep 06 10:39:29.891070 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:29.891038    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd" event={"ID":"9b76b8b8-f168-4f89-8da8-79b479134957","Type":"ContainerStarted","Data":"e9f17ff77faff18ee1380e0ca233f7233a64b660fe2b8c4056f8cd6e6daf1024"}
Sep 06 10:39:29.891518 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:29.891084    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd" event={"ID":"9b76b8b8-f168-4f89-8da8-79b479134957","Type":"ContainerStarted","Data":"f7d57679b75e6b123bd2f3806a3747eb2fd12512f87112ebdc5d7ba0de8eb7a1"}
Sep 06 10:39:29.909193 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:29.909145    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd" podStartSLOduration=1.744162456 podStartE2EDuration="1.909128946s" totalImagesPullingTime="164.96649ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:39:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:39:29.907630768 +0000 UTC m=+582.357411536" watchObservedRunningTime="2026-09-06 10:39:29.909128946 +0000 UTC m=+582.358909714"
Sep 06 10:39:30.521301 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:30.521265    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:39:33.926460 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:33.926427    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"]
Sep 06 10:39:33.930133 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:33.930112    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:33.932443 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:33.932425    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 06 10:39:33.938366 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:33.938343    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"]
Sep 06 10:39:34.001319 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.001286    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b9e9d5f-593e-4538-89aa-5110f37e8eb1-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwwms\" (UID: \"7b9e9d5f-593e-4538-89aa-5110f37e8eb1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:34.001319 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.001320    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b9e9d5f-593e-4538-89aa-5110f37e8eb1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwwms\" (UID: \"7b9e9d5f-593e-4538-89aa-5110f37e8eb1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:34.001544 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.001341    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b9e9d5f-593e-4538-89aa-5110f37e8eb1-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwwms\" (UID: \"7b9e9d5f-593e-4538-89aa-5110f37e8eb1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:34.001544 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.001371    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhfbn\" (UniqueName: \"kubernetes.io/projected/7b9e9d5f-593e-4538-89aa-5110f37e8eb1-kube-api-access-mhfbn\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwwms\" (UID: \"7b9e9d5f-593e-4538-89aa-5110f37e8eb1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:34.001544 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.001429    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b9e9d5f-593e-4538-89aa-5110f37e8eb1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwwms\" (UID: \"7b9e9d5f-593e-4538-89aa-5110f37e8eb1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:34.001544 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.001465    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b9e9d5f-593e-4538-89aa-5110f37e8eb1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwwms\" (UID: \"7b9e9d5f-593e-4538-89aa-5110f37e8eb1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:34.102530 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.102500    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b9e9d5f-593e-4538-89aa-5110f37e8eb1-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwwms\" (UID: \"7b9e9d5f-593e-4538-89aa-5110f37e8eb1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:34.102695 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.102539    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b9e9d5f-593e-4538-89aa-5110f37e8eb1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwwms\" (UID: \"7b9e9d5f-593e-4538-89aa-5110f37e8eb1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:34.102695 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.102568    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b9e9d5f-593e-4538-89aa-5110f37e8eb1-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwwms\" (UID: \"7b9e9d5f-593e-4538-89aa-5110f37e8eb1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:34.102695 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.102649    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhfbn\" (UniqueName: \"kubernetes.io/projected/7b9e9d5f-593e-4538-89aa-5110f37e8eb1-kube-api-access-mhfbn\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwwms\" (UID: \"7b9e9d5f-593e-4538-89aa-5110f37e8eb1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:34.102852 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.102707    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b9e9d5f-593e-4538-89aa-5110f37e8eb1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwwms\" (UID: \"7b9e9d5f-593e-4538-89aa-5110f37e8eb1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:34.102852 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.102769    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b9e9d5f-593e-4538-89aa-5110f37e8eb1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwwms\" (UID: \"7b9e9d5f-593e-4538-89aa-5110f37e8eb1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:34.103131 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.103102    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b9e9d5f-593e-4538-89aa-5110f37e8eb1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwwms\" (UID: \"7b9e9d5f-593e-4538-89aa-5110f37e8eb1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:34.103288 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.103266    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b9e9d5f-593e-4538-89aa-5110f37e8eb1-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwwms\" (UID: \"7b9e9d5f-593e-4538-89aa-5110f37e8eb1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:34.103354 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.103317    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b9e9d5f-593e-4538-89aa-5110f37e8eb1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwwms\" (UID: \"7b9e9d5f-593e-4538-89aa-5110f37e8eb1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:34.105394 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.105365    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b9e9d5f-593e-4538-89aa-5110f37e8eb1-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwwms\" (UID: \"7b9e9d5f-593e-4538-89aa-5110f37e8eb1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:34.105584 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.105564    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b9e9d5f-593e-4538-89aa-5110f37e8eb1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwwms\" (UID: \"7b9e9d5f-593e-4538-89aa-5110f37e8eb1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:34.111195 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.111168    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhfbn\" (UniqueName: \"kubernetes.io/projected/7b9e9d5f-593e-4538-89aa-5110f37e8eb1-kube-api-access-mhfbn\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwwms\" (UID: \"7b9e9d5f-593e-4538-89aa-5110f37e8eb1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:34.240016 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.239956    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:34.508763 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.508739    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"]
Sep 06 10:39:34.510692 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:39:34.510663    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b9e9d5f_593e_4538_89aa_5110f37e8eb1.slice/crio-74b9d503f576a71f58661dc5f1a88797e6068169f150a1d1f36deaaaef842736 WatchSource:0}: Error finding container 74b9d503f576a71f58661dc5f1a88797e6068169f150a1d1f36deaaaef842736: Status 404 returned error can't find the container with id 74b9d503f576a71f58661dc5f1a88797e6068169f150a1d1f36deaaaef842736
Sep 06 10:39:34.921611 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.921579    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms" event={"ID":"7b9e9d5f-593e-4538-89aa-5110f37e8eb1","Type":"ContainerStarted","Data":"f3a183bc65800f1049bad5097395babbd9a89ef67dd9040f83fea16cdfdfb56b"}
Sep 06 10:39:34.921762 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.921620    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms" event={"ID":"7b9e9d5f-593e-4538-89aa-5110f37e8eb1","Type":"ContainerStarted","Data":"74b9d503f576a71f58661dc5f1a88797e6068169f150a1d1f36deaaaef842736"}
Sep 06 10:39:34.938396 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:34.938355    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms" podStartSLOduration=1.753440154 podStartE2EDuration="1.938341596s" totalImagesPullingTime="184.901442ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:39:34.936853557 +0000 UTC m=+587.386634325" watchObservedRunningTime="2026-09-06 10:39:34.938341596 +0000 UTC m=+587.388122363"
Sep 06 10:39:36.124638 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:36.124611    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:39:39.042045 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:39.042015    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:39.042479 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:39.042072    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:39.054343 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:39.054320    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:39.960225 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:39.960197    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dx5fd"
Sep 06 10:39:44.241019 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:44.240986    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:44.241448 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:44.241065    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:44.253473 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:44.253452    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:44.981838 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:44.981811    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwwms"
Sep 06 10:39:48.010677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:48.010632    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/2.log"
Sep 06 10:39:48.013258 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:39:48.013238    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/2.log"
Sep 06 10:40:44.215298 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:40:44.215259    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c6f99d97b-z22xw"]
Sep 06 10:40:44.215864 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:40:44.215506    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7c6f99d97b-z22xw" podUID="f34ebafc-298f-47d3-9af1-5611da648107" containerName="authorino" containerID="cri-o://96f7d34924971acdcc739acc2ead63ce3d3a59545cd7c858a243d5c959b3c931" gracePeriod=30
Sep 06 10:40:44.467203 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:40:44.467140    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c6f99d97b-z22xw"
Sep 06 10:40:44.600236 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:40:44.600202    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f34ebafc-298f-47d3-9af1-5611da648107-tls-cert\") pod \"f34ebafc-298f-47d3-9af1-5611da648107\" (UID: \"f34ebafc-298f-47d3-9af1-5611da648107\") "
Sep 06 10:40:44.600402 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:40:44.600273    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7xvb\" (UniqueName: \"kubernetes.io/projected/f34ebafc-298f-47d3-9af1-5611da648107-kube-api-access-t7xvb\") pod \"f34ebafc-298f-47d3-9af1-5611da648107\" (UID: \"f34ebafc-298f-47d3-9af1-5611da648107\") "
Sep 06 10:40:44.602244 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:40:44.602216    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f34ebafc-298f-47d3-9af1-5611da648107-kube-api-access-t7xvb" (OuterVolumeSpecName: "kube-api-access-t7xvb") pod "f34ebafc-298f-47d3-9af1-5611da648107" (UID: "f34ebafc-298f-47d3-9af1-5611da648107"). InnerVolumeSpecName "kube-api-access-t7xvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:40:44.610542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:40:44.610514    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f34ebafc-298f-47d3-9af1-5611da648107-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f34ebafc-298f-47d3-9af1-5611da648107" (UID: "f34ebafc-298f-47d3-9af1-5611da648107"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 10:40:44.701101 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:40:44.701065    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7xvb\" (UniqueName: \"kubernetes.io/projected/f34ebafc-298f-47d3-9af1-5611da648107-kube-api-access-t7xvb\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:40:44.701101 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:40:44.701095    2557 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f34ebafc-298f-47d3-9af1-5611da648107-tls-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:40:45.262530 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:40:45.262472    2557 generic.go:356] "Generic (PLEG): container finished" podID="f34ebafc-298f-47d3-9af1-5611da648107" containerID="96f7d34924971acdcc739acc2ead63ce3d3a59545cd7c858a243d5c959b3c931" exitCode=0
Sep 06 10:40:45.262530 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:40:45.262525    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c6f99d97b-z22xw"
Sep 06 10:40:45.263025 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:40:45.262533    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c6f99d97b-z22xw" event={"ID":"f34ebafc-298f-47d3-9af1-5611da648107","Type":"ContainerDied","Data":"96f7d34924971acdcc739acc2ead63ce3d3a59545cd7c858a243d5c959b3c931"}
Sep 06 10:40:45.263025 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:40:45.262588    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c6f99d97b-z22xw" event={"ID":"f34ebafc-298f-47d3-9af1-5611da648107","Type":"ContainerDied","Data":"c050a82508b0f88dd052a8cefdaa6f161b8db38e1fb52fa59bbb057c739b9502"}
Sep 06 10:40:45.263025 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:40:45.262614    2557 scope.go:117] "RemoveContainer" containerID="96f7d34924971acdcc739acc2ead63ce3d3a59545cd7c858a243d5c959b3c931"
Sep 06 10:40:45.271142 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:40:45.271121    2557 scope.go:117] "RemoveContainer" containerID="96f7d34924971acdcc739acc2ead63ce3d3a59545cd7c858a243d5c959b3c931"
Sep 06 10:40:45.271428 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:40:45.271406    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96f7d34924971acdcc739acc2ead63ce3d3a59545cd7c858a243d5c959b3c931\": container with ID starting with 96f7d34924971acdcc739acc2ead63ce3d3a59545cd7c858a243d5c959b3c931 not found: ID does not exist" containerID="96f7d34924971acdcc739acc2ead63ce3d3a59545cd7c858a243d5c959b3c931"
Sep 06 10:40:45.271524 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:40:45.271437    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96f7d34924971acdcc739acc2ead63ce3d3a59545cd7c858a243d5c959b3c931"} err="failed to get container status \"96f7d34924971acdcc739acc2ead63ce3d3a59545cd7c858a243d5c959b3c931\": rpc error: code = NotFound desc = could not find container \"96f7d34924971acdcc739acc2ead63ce3d3a59545cd7c858a243d5c959b3c931\": container with ID starting with 96f7d34924971acdcc739acc2ead63ce3d3a59545cd7c858a243d5c959b3c931 not found: ID does not exist"
Sep 06 10:40:45.287074 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:40:45.287038    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c6f99d97b-z22xw"]
Sep 06 10:40:45.288955 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:40:45.288933    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7c6f99d97b-z22xw"]
Sep 06 10:40:46.036409 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:40:46.036372    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f34ebafc-298f-47d3-9af1-5611da648107" path="/var/lib/kubelet/pods/f34ebafc-298f-47d3-9af1-5611da648107/volumes"
Sep 06 10:41:04.537315 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:04.537276    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:41:05.588380 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:05.588344    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7829-140283554823984"]
Sep 06 10:41:05.588778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:05.588744    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f34ebafc-298f-47d3-9af1-5611da648107" containerName="authorino"
Sep 06 10:41:05.588778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:05.588758    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f34ebafc-298f-47d3-9af1-5611da648107" containerName="authorino"
Sep 06 10:41:05.588856 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:05.588828    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f34ebafc-298f-47d3-9af1-5611da648107" containerName="authorino"
Sep 06 10:41:05.593200 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:05.593182    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7829-140283554823984"
Sep 06 10:41:05.595942 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:05.595912    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lwnht\""
Sep 06 10:41:05.597860 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:05.597834    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7829-140283554823984"]
Sep 06 10:41:05.682095 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:05.682059    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlvg5\" (UniqueName: \"kubernetes.io/projected/733dcb45-82ee-4840-8ebc-e8d8506de747-kube-api-access-hlvg5\") pod \"test-curl-7829-140283554823984\" (UID: \"733dcb45-82ee-4840-8ebc-e8d8506de747\") " pod="opendatahub/test-curl-7829-140283554823984"
Sep 06 10:41:05.783769 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:05.783723    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlvg5\" (UniqueName: \"kubernetes.io/projected/733dcb45-82ee-4840-8ebc-e8d8506de747-kube-api-access-hlvg5\") pod \"test-curl-7829-140283554823984\" (UID: \"733dcb45-82ee-4840-8ebc-e8d8506de747\") " pod="opendatahub/test-curl-7829-140283554823984"
Sep 06 10:41:05.792154 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:05.792122    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlvg5\" (UniqueName: \"kubernetes.io/projected/733dcb45-82ee-4840-8ebc-e8d8506de747-kube-api-access-hlvg5\") pod \"test-curl-7829-140283554823984\" (UID: \"733dcb45-82ee-4840-8ebc-e8d8506de747\") " pod="opendatahub/test-curl-7829-140283554823984"
Sep 06 10:41:05.903584 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:05.903478    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7829-140283554823984"
Sep 06 10:41:06.175257 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:06.175226    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7829-140283554823984"]
Sep 06 10:41:06.177447 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:41:06.177419    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod733dcb45_82ee_4840_8ebc_e8d8506de747.slice/crio-95f3cce459f394ca6b5bd047f4079d4a0a648fcd94a8c8ed6ccd94ea84f2164b WatchSource:0}: Error finding container 95f3cce459f394ca6b5bd047f4079d4a0a648fcd94a8c8ed6ccd94ea84f2164b: Status 404 returned error can't find the container with id 95f3cce459f394ca6b5bd047f4079d4a0a648fcd94a8c8ed6ccd94ea84f2164b
Sep 06 10:41:06.179371 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:06.179344    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 10:41:06.364017 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:06.363985    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7829-140283554823984" event={"ID":"733dcb45-82ee-4840-8ebc-e8d8506de747","Type":"ContainerStarted","Data":"95f3cce459f394ca6b5bd047f4079d4a0a648fcd94a8c8ed6ccd94ea84f2164b"}
Sep 06 10:41:08.375723 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:08.375686    2557 generic.go:356] "Generic (PLEG): container finished" podID="733dcb45-82ee-4840-8ebc-e8d8506de747" containerID="1055f911ae802d7fa5ffdf5e631b045057ae5e4718c30d1c272846ade46ec8c3" exitCode=0
Sep 06 10:41:08.376242 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:08.375755    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7829-140283554823984" event={"ID":"733dcb45-82ee-4840-8ebc-e8d8506de747","Type":"ContainerDied","Data":"1055f911ae802d7fa5ffdf5e631b045057ae5e4718c30d1c272846ade46ec8c3"}
Sep 06 10:41:09.161721 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.161691    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"]
Sep 06 10:41:09.164871 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.164848    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.168280 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.168255    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e0690faf-openshift-default-dockercfg-n5zbj\""
Sep 06 10:41:09.186314 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.186281    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"]
Sep 06 10:41:09.317464 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.317431    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-podinfo\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.317645 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.317469    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdf97238-9455-41d4-826b-57cd25e7d39e-istiod-ca-cert\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.317645 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.317509    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-workload-socket\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.317645 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.317535    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-data\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.317645 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.317596    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-envoy\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.317790 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.317656    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-token\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.317790 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.317683    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q58z7\" (UniqueName: \"kubernetes.io/projected/cdf97238-9455-41d4-826b-57cd25e7d39e-kube-api-access-q58z7\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.317790 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.317706    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-workload-certs\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.317790 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.317739    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-credential-socket\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.418453 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.418357    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-workload-socket\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.418453 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.418419    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-data\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.418959 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.418462    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-envoy\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.418959 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.418564    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-token\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.418959 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.418593    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q58z7\" (UniqueName: \"kubernetes.io/projected/cdf97238-9455-41d4-826b-57cd25e7d39e-kube-api-access-q58z7\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.418959 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.418614    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-workload-certs\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.418959 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.418636    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-credential-socket\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.418959 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.418697    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-podinfo\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.418959 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.418726    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdf97238-9455-41d4-826b-57cd25e7d39e-istiod-ca-cert\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.418959 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.418859    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-workload-socket\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.419318 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.418992    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-workload-certs\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.419318 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.419073    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-data\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.419480 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.419453    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdf97238-9455-41d4-826b-57cd25e7d39e-istiod-ca-cert\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.419628 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.419527    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-credential-socket\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.421438 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.421417    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-envoy\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.421570 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.421542    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-podinfo\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.428148 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.428122    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-token\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.428270 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.428254    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q58z7\" (UniqueName: \"kubernetes.io/projected/cdf97238-9455-41d4-826b-57cd25e7d39e-kube-api-access-q58z7\") pod \"e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") " pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.478169 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.478136    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:09.519178 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.518945    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7829-140283554823984"
Sep 06 10:41:09.550728 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.550707    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7829-140283554823984_733dcb45-82ee-4840-8ebc-e8d8506de747/test-curl-7829-140283554823984/0.log"
Sep 06 10:41:09.579498 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.579456    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7829-140283554823984"]
Sep 06 10:41:09.582928 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.582903    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7829-140283554823984"]
Sep 06 10:41:09.620443 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.620421    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlvg5\" (UniqueName: \"kubernetes.io/projected/733dcb45-82ee-4840-8ebc-e8d8506de747-kube-api-access-hlvg5\") pod \"733dcb45-82ee-4840-8ebc-e8d8506de747\" (UID: \"733dcb45-82ee-4840-8ebc-e8d8506de747\") "
Sep 06 10:41:09.622538 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.622515    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/733dcb45-82ee-4840-8ebc-e8d8506de747-kube-api-access-hlvg5" (OuterVolumeSpecName: "kube-api-access-hlvg5") pod "733dcb45-82ee-4840-8ebc-e8d8506de747" (UID: "733dcb45-82ee-4840-8ebc-e8d8506de747"). InnerVolumeSpecName "kube-api-access-hlvg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:41:09.721908 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.721851    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlvg5\" (UniqueName: \"kubernetes.io/projected/733dcb45-82ee-4840-8ebc-e8d8506de747-kube-api-access-hlvg5\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:41:09.755454 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.755433    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"]
Sep 06 10:41:09.757279 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:41:09.757253    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdf97238_9455_41d4_826b_57cd25e7d39e.slice/crio-35048e39a570121a94efafa05f0f87243c953e5895f2f46d3096cad29bf1d4b7 WatchSource:0}: Error finding container 35048e39a570121a94efafa05f0f87243c953e5895f2f46d3096cad29bf1d4b7: Status 404 returned error can't find the container with id 35048e39a570121a94efafa05f0f87243c953e5895f2f46d3096cad29bf1d4b7
Sep 06 10:41:09.759562 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.759533    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:09.759620 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.759595    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:09.759653 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:09.759626    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:10.037187 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.037044    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="733dcb45-82ee-4840-8ebc-e8d8506de747" path="/var/lib/kubelet/pods/733dcb45-82ee-4840-8ebc-e8d8506de747/volumes"
Sep 06 10:41:10.394292 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.394266    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7829-140283554823984"
Sep 06 10:41:10.394292 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.394283    2557 scope.go:117] "RemoveContainer" containerID="1055f911ae802d7fa5ffdf5e631b045057ae5e4718c30d1c272846ade46ec8c3"
Sep 06 10:41:10.396610 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.396580    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4" event={"ID":"cdf97238-9455-41d4-826b-57cd25e7d39e","Type":"ContainerStarted","Data":"83cd292a4823e678124c98fd032ebc4a32fe3189b755f36ba5b30380b02e13ee"}
Sep 06 10:41:10.396904 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.396888    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4" event={"ID":"cdf97238-9455-41d4-826b-57cd25e7d39e","Type":"ContainerStarted","Data":"35048e39a570121a94efafa05f0f87243c953e5895f2f46d3096cad29bf1d4b7"}
Sep 06 10:41:10.420215 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.420192    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"]
Sep 06 10:41:10.420680 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.420664    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="733dcb45-82ee-4840-8ebc-e8d8506de747" containerName="test-curl-7829-140283554823984"
Sep 06 10:41:10.420680 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.420681    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="733dcb45-82ee-4840-8ebc-e8d8506de747" containerName="test-curl-7829-140283554823984"
Sep 06 10:41:10.420793 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.420742    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="733dcb45-82ee-4840-8ebc-e8d8506de747" containerName="test-curl-7829-140283554823984"
Sep 06 10:41:10.429248 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.429201    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4" podStartSLOduration=1.429183598 podStartE2EDuration="1.429183598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:41:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:41:10.42731349 +0000 UTC m=+682.877094259" watchObservedRunningTime="2026-09-06 10:41:10.429183598 +0000 UTC m=+682.878964367"
Sep 06 10:41:10.432545 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.432528    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.435933 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.435909    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-55111c-openshift-default-dockercfg-qlbsf\""
Sep 06 10:41:10.442138 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.442118    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"]
Sep 06 10:41:10.478649 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.478621    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:10.483621 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.483600    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:10.526646 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.526624    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-istio-envoy\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.526742 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.526651    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-workload-socket\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.526742 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.526669    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxf45\" (UniqueName: \"kubernetes.io/projected/83ae9b0d-c656-429b-a014-d09481616a67-kube-api-access-fxf45\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.526742 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.526689    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-workload-certs\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.526742 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.526722    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-istio-data\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.526897 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.526745    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83ae9b0d-c656-429b-a014-d09481616a67-istio-token\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.526897 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.526800    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-credential-socket\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.526897 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.526883    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83ae9b0d-c656-429b-a014-d09481616a67-istio-podinfo\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.527000 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.526905    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83ae9b0d-c656-429b-a014-d09481616a67-istiod-ca-cert\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.627942 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.627909    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-istio-envoy\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.628113 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.627951    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-workload-socket\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.628113 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.627976    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxf45\" (UniqueName: \"kubernetes.io/projected/83ae9b0d-c656-429b-a014-d09481616a67-kube-api-access-fxf45\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.628113 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.628023    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-workload-certs\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.628113 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.628081    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-istio-data\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.628113 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.628113    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83ae9b0d-c656-429b-a014-d09481616a67-istio-token\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.628365 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.628149    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-credential-socket\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.628421 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.628386    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83ae9b0d-c656-429b-a014-d09481616a67-istio-podinfo\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.628473 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.628434    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83ae9b0d-c656-429b-a014-d09481616a67-istiod-ca-cert\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.628473 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.628441    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-workload-certs\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.628597 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.628534    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-istio-data\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.628597 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.628587    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-workload-socket\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.628754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.628735    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-credential-socket\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.629104 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.629082    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83ae9b0d-c656-429b-a014-d09481616a67-istiod-ca-cert\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.630308 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.630282    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-istio-envoy\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.630752 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.630729    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83ae9b0d-c656-429b-a014-d09481616a67-istio-podinfo\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.636329 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.636299    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83ae9b0d-c656-429b-a014-d09481616a67-istio-token\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.636770 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.636752    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxf45\" (UniqueName: \"kubernetes.io/projected/83ae9b0d-c656-429b-a014-d09481616a67-kube-api-access-fxf45\") pod \"e2e-autores-55111c-openshift-default-84675d875d-f58sc\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") " pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:10.744193 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:10.744136    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:11.024377 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:11.024348    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"]
Sep 06 10:41:11.025355 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:41:11.025328    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83ae9b0d_c656_429b_a014_d09481616a67.slice/crio-5479b8ac1c6b276fca179da80b5f4185e9b2d500bdf1842799172a7b78c638c3 WatchSource:0}: Error finding container 5479b8ac1c6b276fca179da80b5f4185e9b2d500bdf1842799172a7b78c638c3: Status 404 returned error can't find the container with id 5479b8ac1c6b276fca179da80b5f4185e9b2d500bdf1842799172a7b78c638c3
Sep 06 10:41:11.027644 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:11.027607    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:11.027747 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:11.027688    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:11.027747 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:11.027731    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:11.403919 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:11.403880    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc" event={"ID":"83ae9b0d-c656-429b-a014-d09481616a67","Type":"ContainerStarted","Data":"a017d0f330abe2326600a3f00c7cbb43db5e674e91f9980659e264d6af8dd369"}
Sep 06 10:41:11.404056 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:11.403945    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc" event={"ID":"83ae9b0d-c656-429b-a014-d09481616a67","Type":"ContainerStarted","Data":"5479b8ac1c6b276fca179da80b5f4185e9b2d500bdf1842799172a7b78c638c3"}
Sep 06 10:41:11.404125 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:11.404112    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:11.405100 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:11.405077    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:11.424826 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:11.424787    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc" podStartSLOduration=1.4247741010000001 podStartE2EDuration="1.424774101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:41:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:41:11.423820221 +0000 UTC m=+683.873600991" watchObservedRunningTime="2026-09-06 10:41:11.424774101 +0000 UTC m=+683.874554868"
Sep 06 10:41:11.745037 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:11.744965    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:12.749942 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:12.749906    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:13.416357 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:13.416325    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:13.417126 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:13.417104    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:19.132569 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:19.132536    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:41:29.457671 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.457612    2557 patch_prober.go:28] interesting pod/e2e-autores-55111c-openshift-default-84675d875d-f58sc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 06 10:41:29.458056 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.457696    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc" podUID="83ae9b0d-c656-429b-a014-d09481616a67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 06 10:41:29.719312 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.719222    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"]
Sep 06 10:41:29.722444 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.722424    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.724979 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.724958    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-636b04-openshift-default-dockercfg-8fmt7\""
Sep 06 10:41:29.736230 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.736209    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"]
Sep 06 10:41:29.806754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.806685    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-envoy\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.806863 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.806758    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-workload-certs\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.806863 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.806791    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-token\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.806863 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.806820    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d55r4\" (UniqueName: \"kubernetes.io/projected/5dc1d481-5896-4ce2-8360-ab174e11945a-kube-api-access-d55r4\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.806968 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.806921    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-workload-socket\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.806968 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.806951    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-credential-socket\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.807036 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.806970    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-podinfo\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.807071 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.807041    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dc1d481-5896-4ce2-8360-ab174e11945a-istiod-ca-cert\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.807103 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.807078    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-data\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.908127 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.908090    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-credential-socket\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.908305 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.908132    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-podinfo\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.908305 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.908265    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dc1d481-5896-4ce2-8360-ab174e11945a-istiod-ca-cert\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.908430 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.908320    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-data\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.908430 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.908396    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-envoy\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.908577 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.908472    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-workload-certs\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.908632 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.908596    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-credential-socket\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.908754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.908643    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-token\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.908754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.908677    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d55r4\" (UniqueName: \"kubernetes.io/projected/5dc1d481-5896-4ce2-8360-ab174e11945a-kube-api-access-d55r4\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.908851 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.908766    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-workload-socket\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.908851 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.908773    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-workload-certs\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.908952 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.908852    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-data\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.909129 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.909105    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-workload-socket\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.909200 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.909138    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dc1d481-5896-4ce2-8360-ab174e11945a-istiod-ca-cert\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.910649 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.910626    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-envoy\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.910784 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.910766    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-podinfo\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.919001 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.918975    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-token\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:29.919264 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:29.919244    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d55r4\" (UniqueName: \"kubernetes.io/projected/5dc1d481-5896-4ce2-8360-ab174e11945a-kube-api-access-d55r4\") pod \"e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") " pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:30.037754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:30.037681    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:30.319091 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:30.319064    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"]
Sep 06 10:41:30.320931 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:41:30.320900    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dc1d481_5896_4ce2_8360_ab174e11945a.slice/crio-719ca724b7dfd5cbcc2316762af553a052afc43eb226dffa506550d72dc64e66 WatchSource:0}: Error finding container 719ca724b7dfd5cbcc2316762af553a052afc43eb226dffa506550d72dc64e66: Status 404 returned error can't find the container with id 719ca724b7dfd5cbcc2316762af553a052afc43eb226dffa506550d72dc64e66
Sep 06 10:41:30.323343 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:30.323305    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:30.323454 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:30.323434    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:30.323536 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:30.323484    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:30.519254 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:30.519216    2557 generic.go:356] "Generic (PLEG): container finished" podID="83ae9b0d-c656-429b-a014-d09481616a67" containerID="a017d0f330abe2326600a3f00c7cbb43db5e674e91f9980659e264d6af8dd369" exitCode=0
Sep 06 10:41:30.519697 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:30.519295    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc" event={"ID":"83ae9b0d-c656-429b-a014-d09481616a67","Type":"ContainerDied","Data":"a017d0f330abe2326600a3f00c7cbb43db5e674e91f9980659e264d6af8dd369"}
Sep 06 10:41:30.519821 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:30.519802    2557 scope.go:117] "RemoveContainer" containerID="a017d0f330abe2326600a3f00c7cbb43db5e674e91f9980659e264d6af8dd369"
Sep 06 10:41:30.521138 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:30.521102    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:30.521253 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:30.521196    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:30.521253 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:30.521242    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:30.521547 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:30.521527    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z" event={"ID":"5dc1d481-5896-4ce2-8360-ab174e11945a","Type":"ContainerStarted","Data":"3d3828292538bf177f5f5e75ebc026253c5d9c2dcf02184705e5a3e606a8ed04"}
Sep 06 10:41:30.521630 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:30.521572    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z" event={"ID":"5dc1d481-5896-4ce2-8360-ab174e11945a","Type":"ContainerStarted","Data":"719ca724b7dfd5cbcc2316762af553a052afc43eb226dffa506550d72dc64e66"}
Sep 06 10:41:30.563773 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:30.563704    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z" podStartSLOduration=1.5636828619999998 podStartE2EDuration="1.563682862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:41:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:41:30.561989025 +0000 UTC m=+703.011769823" watchObservedRunningTime="2026-09-06 10:41:30.563682862 +0000 UTC m=+703.013463633"
Sep 06 10:41:31.038893 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:31.038865    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:31.043621 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:31.043603    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:31.528324 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:31.528290    2557 generic.go:356] "Generic (PLEG): container finished" podID="cdf97238-9455-41d4-826b-57cd25e7d39e" containerID="83cd292a4823e678124c98fd032ebc4a32fe3189b755f36ba5b30380b02e13ee" exitCode=0
Sep 06 10:41:31.528819 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:31.528366    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4" event={"ID":"cdf97238-9455-41d4-826b-57cd25e7d39e","Type":"ContainerDied","Data":"83cd292a4823e678124c98fd032ebc4a32fe3189b755f36ba5b30380b02e13ee"}
Sep 06 10:41:31.528881 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:31.528853    2557 scope.go:117] "RemoveContainer" containerID="83cd292a4823e678124c98fd032ebc4a32fe3189b755f36ba5b30380b02e13ee"
Sep 06 10:41:31.530357 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:31.530321    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:31.530468 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:31.530418    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:31.530535 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:31.530468    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:31.530834 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:31.530807    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc" event={"ID":"83ae9b0d-c656-429b-a014-d09481616a67","Type":"ContainerStarted","Data":"3869f329a65f3734c76fc237b476920feca4cab30ba522baef613d56a15a710e"}
Sep 06 10:41:31.531126 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:31.531112    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:31.532791 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:31.532742    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:31.745312 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:31.744902    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:31.745312 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:31.745024    2557 patch_prober.go:28] interesting pod/e2e-autores-55111c-openshift-default-84675d875d-f58sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 06 10:41:31.745312 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:31.745066    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc" podUID="83ae9b0d-c656-429b-a014-d09481616a67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 06 10:41:32.537300 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:32.537259    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4" event={"ID":"cdf97238-9455-41d4-826b-57cd25e7d39e","Type":"ContainerStarted","Data":"be0336df3432f29854dc48d0ab043f6a594d306bbec8519ba400356762a04afa"}
Sep 06 10:41:32.745371 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:32.745335    2557 patch_prober.go:28] interesting pod/e2e-autores-55111c-openshift-default-84675d875d-f58sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 06 10:41:32.745558 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:32.745401    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc" podUID="83ae9b0d-c656-429b-a014-d09481616a67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 06 10:41:33.478819 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:33.478778    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:33.479028 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:33.478907    2557 patch_prober.go:28] interesting pod/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 06 10:41:33.479028 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:33.478953    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4" podUID="cdf97238-9455-41d4-826b-57cd25e7d39e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 06 10:41:33.745017 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:33.744939    2557 patch_prober.go:28] interesting pod/e2e-autores-55111c-openshift-default-84675d875d-f58sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 06 10:41:33.745436 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:33.745406    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc" podUID="83ae9b0d-c656-429b-a014-d09481616a67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 06 10:41:34.479144 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:34.479104    2557 patch_prober.go:28] interesting pod/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 06 10:41:34.479353 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:34.479187    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4" podUID="cdf97238-9455-41d4-826b-57cd25e7d39e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 06 10:41:34.744815 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:34.744727    2557 patch_prober.go:28] interesting pod/e2e-autores-55111c-openshift-default-84675d875d-f58sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 06 10:41:34.744815 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:34.744784    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc" podUID="83ae9b0d-c656-429b-a014-d09481616a67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 06 10:41:35.479449 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:35.479404    2557 patch_prober.go:28] interesting pod/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 06 10:41:35.479964 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:35.479477    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4" podUID="cdf97238-9455-41d4-826b-57cd25e7d39e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 06 10:41:35.745356 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:35.745261    2557 patch_prober.go:28] interesting pod/e2e-autores-55111c-openshift-default-84675d875d-f58sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 06 10:41:35.745356 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:35.745320    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc" podUID="83ae9b0d-c656-429b-a014-d09481616a67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 06 10:41:36.479358 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:36.479284    2557 patch_prober.go:28] interesting pod/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 06 10:41:36.480081 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:36.480012    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4" podUID="cdf97238-9455-41d4-826b-57cd25e7d39e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 06 10:41:36.744592 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:36.744506    2557 patch_prober.go:28] interesting pod/e2e-autores-55111c-openshift-default-84675d875d-f58sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 06 10:41:36.744592 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:36.744568    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc" podUID="83ae9b0d-c656-429b-a014-d09481616a67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 06 10:41:37.245813 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:37.245776    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0690faf-p6csp"]
Sep 06 10:41:37.249720 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:37.249702    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0690faf-p6csp"
Sep 06 10:41:37.252163 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:37.252144    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 10:41:37.252257 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:37.252164    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\""
Sep 06 10:41:37.256360 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:37.256326    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0690faf-p6csp"]
Sep 06 10:41:37.289048 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:37.289005    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c967c82f-a2a6-4849-b654-fec2fbf73bc9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e0690faf-p6csp\" (UID: \"c967c82f-a2a6-4849-b654-fec2fbf73bc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0690faf-p6csp"
Sep 06 10:41:37.389931 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:37.389893    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c967c82f-a2a6-4849-b654-fec2fbf73bc9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e0690faf-p6csp\" (UID: \"c967c82f-a2a6-4849-b654-fec2fbf73bc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0690faf-p6csp"
Sep 06 10:41:37.390681 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:37.390654    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c967c82f-a2a6-4849-b654-fec2fbf73bc9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e0690faf-p6csp\" (UID: \"c967c82f-a2a6-4849-b654-fec2fbf73bc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0690faf-p6csp"
Sep 06 10:41:37.479358 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:37.479316    2557 patch_prober.go:28] interesting pod/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 06 10:41:37.479570 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:37.479382    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4" podUID="cdf97238-9455-41d4-826b-57cd25e7d39e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 06 10:41:37.561625 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:37.561553    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0690faf-p6csp"
Sep 06 10:41:37.748198 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:37.748169    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:37.748452 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:37.748423    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:37.749093 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:37.749075    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:41:37.853369 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:37.853333    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0690faf-p6csp"]
Sep 06 10:41:37.853960 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:41:37.853923    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc967c82f_a2a6_4849_b654_fec2fbf73bc9.slice/crio-069da030e5dd0209869575b0356df50e7c1dcb734b5d7a9c91d96813e8532916 WatchSource:0}: Error finding container 069da030e5dd0209869575b0356df50e7c1dcb734b5d7a9c91d96813e8532916: Status 404 returned error can't find the container with id 069da030e5dd0209869575b0356df50e7c1dcb734b5d7a9c91d96813e8532916
Sep 06 10:41:38.479410 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:38.479371    2557 patch_prober.go:28] interesting pod/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 06 10:41:38.479624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:38.479446    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4" podUID="cdf97238-9455-41d4-826b-57cd25e7d39e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 06 10:41:38.572047 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:38.572001    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0690faf-p6csp" event={"ID":"c967c82f-a2a6-4849-b654-fec2fbf73bc9","Type":"ContainerStarted","Data":"069da030e5dd0209869575b0356df50e7c1dcb734b5d7a9c91d96813e8532916"}
Sep 06 10:41:39.483968 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:39.483936    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:39.484367 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:39.484334    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:39.485101 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:39.485080    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:40.586391 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:40.586316    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0690faf-p6csp" event={"ID":"c967c82f-a2a6-4849-b654-fec2fbf73bc9","Type":"ContainerStarted","Data":"d4b19881a0ac8af76623765a5aa9d7b71988652a1342308298e973435a52998a"}
Sep 06 10:41:40.605846 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:40.605804    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0690faf-p6csp" podStartSLOduration=1.314525981 podStartE2EDuration="3.60579158s" totalImagesPullingTime="2.291265599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:41:40.602565078 +0000 UTC m=+713.052345847" watchObservedRunningTime="2026-09-06 10:41:40.60579158 +0000 UTC m=+713.055572349"
Sep 06 10:41:41.060173 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:41.060144    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:41:41.594139 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:41.594098    2557 generic.go:356] "Generic (PLEG): container finished" podID="c967c82f-a2a6-4849-b654-fec2fbf73bc9" containerID="d4b19881a0ac8af76623765a5aa9d7b71988652a1342308298e973435a52998a" exitCode=0
Sep 06 10:41:41.594550 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:41.594177    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0690faf-p6csp" event={"ID":"c967c82f-a2a6-4849-b654-fec2fbf73bc9","Type":"ContainerDied","Data":"d4b19881a0ac8af76623765a5aa9d7b71988652a1342308298e973435a52998a"}
Sep 06 10:41:42.754762 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:42.754737    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0690faf-p6csp"
Sep 06 10:41:42.848082 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:42.848049    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c967c82f-a2a6-4849-b654-fec2fbf73bc9-maas-api-service-ca\") pod \"c967c82f-a2a6-4849-b654-fec2fbf73bc9\" (UID: \"c967c82f-a2a6-4849-b654-fec2fbf73bc9\") "
Sep 06 10:41:42.848399 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:42.848378    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c967c82f-a2a6-4849-b654-fec2fbf73bc9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c967c82f-a2a6-4849-b654-fec2fbf73bc9" (UID: "c967c82f-a2a6-4849-b654-fec2fbf73bc9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:41:42.949753 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:42.949671    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c967c82f-a2a6-4849-b654-fec2fbf73bc9-maas-api-service-ca\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:41:43.606285 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:43.606260    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0690faf-p6csp"
Sep 06 10:41:43.606473 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:43.606290    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0690faf-p6csp" event={"ID":"c967c82f-a2a6-4849-b654-fec2fbf73bc9","Type":"ContainerDied","Data":"069da030e5dd0209869575b0356df50e7c1dcb734b5d7a9c91d96813e8532916"}
Sep 06 10:41:43.606473 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:43.606337    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="069da030e5dd0209869575b0356df50e7c1dcb734b5d7a9c91d96813e8532916"
Sep 06 10:41:51.652156 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:51.652121    2557 generic.go:356] "Generic (PLEG): container finished" podID="5dc1d481-5896-4ce2-8360-ab174e11945a" containerID="3d3828292538bf177f5f5e75ebc026253c5d9c2dcf02184705e5a3e606a8ed04" exitCode=0
Sep 06 10:41:51.652603 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:51.652200    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z" event={"ID":"5dc1d481-5896-4ce2-8360-ab174e11945a","Type":"ContainerDied","Data":"3d3828292538bf177f5f5e75ebc026253c5d9c2dcf02184705e5a3e606a8ed04"}
Sep 06 10:41:51.652654 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:51.652630    2557 scope.go:117] "RemoveContainer" containerID="3d3828292538bf177f5f5e75ebc026253c5d9c2dcf02184705e5a3e606a8ed04"
Sep 06 10:41:51.653719 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:51.653689    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:51.653774 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:51.653757    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:51.653821 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:51.653788    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:52.659804 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:52.659767    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z" event={"ID":"5dc1d481-5896-4ce2-8360-ab174e11945a","Type":"ContainerStarted","Data":"feccd848d63f7185a2a6181b8c8dcc6b581254c6febebdcf9348e3f406b26318"}
Sep 06 10:41:53.038789 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:53.038697    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:53.038958 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:53.038917    2557 patch_prober.go:28] interesting pod/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 06 10:41:53.039026 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:53.038972    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z" podUID="5dc1d481-5896-4ce2-8360-ab174e11945a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 06 10:41:54.003766 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:54.003728    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0690faf-p6csp"]
Sep 06 10:41:54.007740 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:54.007715    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0690faf-p6csp"]
Sep 06 10:41:54.037431 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:54.037391    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c967c82f-a2a6-4849-b654-fec2fbf73bc9" path="/var/lib/kubelet/pods/c967c82f-a2a6-4849-b654-fec2fbf73bc9/volumes"
Sep 06 10:41:54.038019 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:54.037995    2557 patch_prober.go:28] interesting pod/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 06 10:41:54.038124 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:54.038049    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z" podUID="5dc1d481-5896-4ce2-8360-ab174e11945a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 06 10:41:54.149046 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:54.149015    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"]
Sep 06 10:41:54.149286 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:54.149262    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4" podUID="cdf97238-9455-41d4-826b-57cd25e7d39e" containerName="istio-proxy" containerID="cri-o://be0336df3432f29854dc48d0ab043f6a594d306bbec8519ba400356762a04afa" gracePeriod=30
Sep 06 10:41:54.489816 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:54.489776    2557 patch_prober.go:28] interesting pod/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 06 10:41:54.490011 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:54.489850    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4" podUID="cdf97238-9455-41d4-826b-57cd25e7d39e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 10:41:55.038430 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.038394    2557 patch_prober.go:28] interesting pod/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 06 10:41:55.038823 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.038451    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z" podUID="5dc1d481-5896-4ce2-8360-ab174e11945a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 06 10:41:55.129768 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.129735    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"]
Sep 06 10:41:55.130191 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.130176    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c967c82f-a2a6-4849-b654-fec2fbf73bc9" containerName="revoke-keys"
Sep 06 10:41:55.130267 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.130192    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c967c82f-a2a6-4849-b654-fec2fbf73bc9" containerName="revoke-keys"
Sep 06 10:41:55.130304 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.130268    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c967c82f-a2a6-4849-b654-fec2fbf73bc9" containerName="revoke-keys"
Sep 06 10:41:55.132640 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.132621    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.135159 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.135139    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-2dae43f5-gw-openshift-default-dockercfg-4s4r9\""
Sep 06 10:41:55.143644 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.143622    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"]
Sep 06 10:41:55.280515 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.279979    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-istio-data\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.280515 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.280037    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rwjj\" (UniqueName: \"kubernetes.io/projected/b1d3bd95-218f-44e4-9445-79df9070863f-kube-api-access-5rwjj\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.280515 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.280081    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1d3bd95-218f-44e4-9445-79df9070863f-istiod-ca-cert\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.280515 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.280107    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-credential-socket\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.280515 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.280158    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-workload-socket\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.280515 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.280184    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1d3bd95-218f-44e4-9445-79df9070863f-istio-token\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.280515 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.280230    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-istio-envoy\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.280515 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.280287    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1d3bd95-218f-44e4-9445-79df9070863f-istio-podinfo\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.280515 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.280329    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-workload-certs\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.382122 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.381234    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1d3bd95-218f-44e4-9445-79df9070863f-istio-podinfo\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.382122 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.381301    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-workload-certs\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.382122 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.381357    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-istio-data\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.382122 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.381391    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rwjj\" (UniqueName: \"kubernetes.io/projected/b1d3bd95-218f-44e4-9445-79df9070863f-kube-api-access-5rwjj\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.382122 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.381434    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1d3bd95-218f-44e4-9445-79df9070863f-istiod-ca-cert\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.382122 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.381462    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-credential-socket\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.382122 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.381531    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-workload-socket\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.382122 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.381559    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1d3bd95-218f-44e4-9445-79df9070863f-istio-token\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.382122 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.381608    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-istio-envoy\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.384640 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.383762    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-workload-certs\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.384640 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.383919    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-credential-socket\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.384640 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.384000    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-istio-data\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.384640 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.384257    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-workload-socket\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.384640 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.384600    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1d3bd95-218f-44e4-9445-79df9070863f-istiod-ca-cert\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.385242 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.385198    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-istio-envoy\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.389054 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.389031    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1d3bd95-218f-44e4-9445-79df9070863f-istio-podinfo\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.395170 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.394990    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1d3bd95-218f-44e4-9445-79df9070863f-istio-token\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.395170 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.395123    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rwjj\" (UniqueName: \"kubernetes.io/projected/b1d3bd95-218f-44e4-9445-79df9070863f-kube-api-access-5rwjj\") pod \"e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") " pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.444094 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.444062    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:55.743519 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.743448    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"]
Sep 06 10:41:55.745603 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:41:55.745578    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1d3bd95_218f_44e4_9445_79df9070863f.slice/crio-5aa93c4599c6df146564b65f8330b423522d6acdd3b0847ad76688c41ad6d303 WatchSource:0}: Error finding container 5aa93c4599c6df146564b65f8330b423522d6acdd3b0847ad76688c41ad6d303: Status 404 returned error can't find the container with id 5aa93c4599c6df146564b65f8330b423522d6acdd3b0847ad76688c41ad6d303
Sep 06 10:41:55.748051 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.748019    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:55.748137 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.748080    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:55.748137 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:55.748110    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:41:56.038435 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:56.038358    2557 patch_prober.go:28] interesting pod/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 06 10:41:56.038435 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:56.038413    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z" podUID="5dc1d481-5896-4ce2-8360-ab174e11945a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 06 10:41:56.695259 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:56.695221    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh" event={"ID":"b1d3bd95-218f-44e4-9445-79df9070863f","Type":"ContainerStarted","Data":"75925f9516f408b95cedbb6a0f3f4b8d31c385632b3a39dd4d4f5eeab78cee5f"}
Sep 06 10:41:56.695442 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:56.695303    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh" event={"ID":"b1d3bd95-218f-44e4-9445-79df9070863f","Type":"ContainerStarted","Data":"5aa93c4599c6df146564b65f8330b423522d6acdd3b0847ad76688c41ad6d303"}
Sep 06 10:41:56.715318 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:56.715269    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh" podStartSLOduration=1.7152565530000001 podStartE2EDuration="1.715256553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:41:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:41:56.713261659 +0000 UTC m=+729.163042426" watchObservedRunningTime="2026-09-06 10:41:56.715256553 +0000 UTC m=+729.165037378"
Sep 06 10:41:57.038419 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:57.038335    2557 patch_prober.go:28] interesting pod/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 06 10:41:57.038419 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:57.038389    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z" podUID="5dc1d481-5896-4ce2-8360-ab174e11945a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 06 10:41:57.444481 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:57.444450    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:57.449745 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:57.449718    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:57.700605 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:57.700503    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:57.701359 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:57.701342    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:41:58.041794 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:58.041722    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:58.042044 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:58.042021    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:58.042784 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:58.042762    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:41:59.412070 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.412046    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:59.428788 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.428729    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-token\") pod \"cdf97238-9455-41d4-826b-57cd25e7d39e\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") "
Sep 06 10:41:59.428903 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.428793    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-data\") pod \"cdf97238-9455-41d4-826b-57cd25e7d39e\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") "
Sep 06 10:41:59.428903 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.428816    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-workload-certs\") pod \"cdf97238-9455-41d4-826b-57cd25e7d39e\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") "
Sep 06 10:41:59.428903 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.428839    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q58z7\" (UniqueName: \"kubernetes.io/projected/cdf97238-9455-41d4-826b-57cd25e7d39e-kube-api-access-q58z7\") pod \"cdf97238-9455-41d4-826b-57cd25e7d39e\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") "
Sep 06 10:41:59.428903 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.428854    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-credential-socket\") pod \"cdf97238-9455-41d4-826b-57cd25e7d39e\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") "
Sep 06 10:41:59.428903 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.428880    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdf97238-9455-41d4-826b-57cd25e7d39e-istiod-ca-cert\") pod \"cdf97238-9455-41d4-826b-57cd25e7d39e\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") "
Sep 06 10:41:59.429162 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.428938    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-envoy\") pod \"cdf97238-9455-41d4-826b-57cd25e7d39e\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") "
Sep 06 10:41:59.429162 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.429016    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-podinfo\") pod \"cdf97238-9455-41d4-826b-57cd25e7d39e\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") "
Sep 06 10:41:59.429162 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.429040    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-workload-socket\") pod \"cdf97238-9455-41d4-826b-57cd25e7d39e\" (UID: \"cdf97238-9455-41d4-826b-57cd25e7d39e\") "
Sep 06 10:41:59.429322 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.429150    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-data" (OuterVolumeSpecName: "istio-data") pod "cdf97238-9455-41d4-826b-57cd25e7d39e" (UID: "cdf97238-9455-41d4-826b-57cd25e7d39e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:41:59.429322 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.429159    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cdf97238-9455-41d4-826b-57cd25e7d39e" (UID: "cdf97238-9455-41d4-826b-57cd25e7d39e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:41:59.429322 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.429186    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cdf97238-9455-41d4-826b-57cd25e7d39e" (UID: "cdf97238-9455-41d4-826b-57cd25e7d39e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:41:59.429483 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.429369    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cdf97238-9455-41d4-826b-57cd25e7d39e" (UID: "cdf97238-9455-41d4-826b-57cd25e7d39e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:41:59.429483 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.429412    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-data\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:41:59.429483 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.429429    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-workload-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:41:59.429483 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.429444    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-credential-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:41:59.429483 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.429466    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdf97238-9455-41d4-826b-57cd25e7d39e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cdf97238-9455-41d4-826b-57cd25e7d39e" (UID: "cdf97238-9455-41d4-826b-57cd25e7d39e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:41:59.431124 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.431082    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cdf97238-9455-41d4-826b-57cd25e7d39e" (UID: "cdf97238-9455-41d4-826b-57cd25e7d39e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 10:41:59.431124 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.431119    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cdf97238-9455-41d4-826b-57cd25e7d39e" (UID: "cdf97238-9455-41d4-826b-57cd25e7d39e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:41:59.431327 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.431309    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-token" (OuterVolumeSpecName: "istio-token") pod "cdf97238-9455-41d4-826b-57cd25e7d39e" (UID: "cdf97238-9455-41d4-826b-57cd25e7d39e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:41:59.431373 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.431333    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdf97238-9455-41d4-826b-57cd25e7d39e-kube-api-access-q58z7" (OuterVolumeSpecName: "kube-api-access-q58z7") pod "cdf97238-9455-41d4-826b-57cd25e7d39e" (UID: "cdf97238-9455-41d4-826b-57cd25e7d39e"). InnerVolumeSpecName "kube-api-access-q58z7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:41:59.530410 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.530376    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-envoy\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:41:59.530410 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.530403    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-podinfo\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:41:59.530410 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.530412    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf97238-9455-41d4-826b-57cd25e7d39e-workload-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:41:59.530410 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.530420    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdf97238-9455-41d4-826b-57cd25e7d39e-istio-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:41:59.530782 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.530429    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q58z7\" (UniqueName: \"kubernetes.io/projected/cdf97238-9455-41d4-826b-57cd25e7d39e-kube-api-access-q58z7\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:41:59.530782 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.530437    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdf97238-9455-41d4-826b-57cd25e7d39e-istiod-ca-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:41:59.712405 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.712307    2557 generic.go:356] "Generic (PLEG): container finished" podID="cdf97238-9455-41d4-826b-57cd25e7d39e" containerID="be0336df3432f29854dc48d0ab043f6a594d306bbec8519ba400356762a04afa" exitCode=0
Sep 06 10:41:59.712405 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.712374    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"
Sep 06 10:41:59.712405 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.712388    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4" event={"ID":"cdf97238-9455-41d4-826b-57cd25e7d39e","Type":"ContainerDied","Data":"be0336df3432f29854dc48d0ab043f6a594d306bbec8519ba400356762a04afa"}
Sep 06 10:41:59.712658 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.712474    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4" event={"ID":"cdf97238-9455-41d4-826b-57cd25e7d39e","Type":"ContainerDied","Data":"35048e39a570121a94efafa05f0f87243c953e5895f2f46d3096cad29bf1d4b7"}
Sep 06 10:41:59.712658 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.712512    2557 scope.go:117] "RemoveContainer" containerID="be0336df3432f29854dc48d0ab043f6a594d306bbec8519ba400356762a04afa"
Sep 06 10:41:59.722290 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.722268    2557 scope.go:117] "RemoveContainer" containerID="83cd292a4823e678124c98fd032ebc4a32fe3189b755f36ba5b30380b02e13ee"
Sep 06 10:41:59.729970 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.729948    2557 scope.go:117] "RemoveContainer" containerID="be0336df3432f29854dc48d0ab043f6a594d306bbec8519ba400356762a04afa"
Sep 06 10:41:59.730206 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:41:59.730189    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be0336df3432f29854dc48d0ab043f6a594d306bbec8519ba400356762a04afa\": container with ID starting with be0336df3432f29854dc48d0ab043f6a594d306bbec8519ba400356762a04afa not found: ID does not exist" containerID="be0336df3432f29854dc48d0ab043f6a594d306bbec8519ba400356762a04afa"
Sep 06 10:41:59.730254 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.730213    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be0336df3432f29854dc48d0ab043f6a594d306bbec8519ba400356762a04afa"} err="failed to get container status \"be0336df3432f29854dc48d0ab043f6a594d306bbec8519ba400356762a04afa\": rpc error: code = NotFound desc = could not find container \"be0336df3432f29854dc48d0ab043f6a594d306bbec8519ba400356762a04afa\": container with ID starting with be0336df3432f29854dc48d0ab043f6a594d306bbec8519ba400356762a04afa not found: ID does not exist"
Sep 06 10:41:59.730254 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.730228    2557 scope.go:117] "RemoveContainer" containerID="83cd292a4823e678124c98fd032ebc4a32fe3189b755f36ba5b30380b02e13ee"
Sep 06 10:41:59.730479 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:41:59.730452    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83cd292a4823e678124c98fd032ebc4a32fe3189b755f36ba5b30380b02e13ee\": container with ID starting with 83cd292a4823e678124c98fd032ebc4a32fe3189b755f36ba5b30380b02e13ee not found: ID does not exist" containerID="83cd292a4823e678124c98fd032ebc4a32fe3189b755f36ba5b30380b02e13ee"
Sep 06 10:41:59.730535 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.730504    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83cd292a4823e678124c98fd032ebc4a32fe3189b755f36ba5b30380b02e13ee"} err="failed to get container status \"83cd292a4823e678124c98fd032ebc4a32fe3189b755f36ba5b30380b02e13ee\": rpc error: code = NotFound desc = could not find container \"83cd292a4823e678124c98fd032ebc4a32fe3189b755f36ba5b30380b02e13ee\": container with ID starting with 83cd292a4823e678124c98fd032ebc4a32fe3189b755f36ba5b30380b02e13ee not found: ID does not exist"
Sep 06 10:41:59.737688 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.737666    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"]
Sep 06 10:41:59.741401 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:41:59.741380    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e0690faf-openshift-default-6c7fc99f4b-5vhg4"]
Sep 06 10:42:00.038010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:00.037921    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdf97238-9455-41d4-826b-57cd25e7d39e" path="/var/lib/kubelet/pods/cdf97238-9455-41d4-826b-57cd25e7d39e/volumes"
Sep 06 10:42:00.114876 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:00.114841    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:42:18.828728 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:18.828625    2557 generic.go:356] "Generic (PLEG): container finished" podID="b1d3bd95-218f-44e4-9445-79df9070863f" containerID="75925f9516f408b95cedbb6a0f3f4b8d31c385632b3a39dd4d4f5eeab78cee5f" exitCode=0
Sep 06 10:42:18.828728 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:18.828681    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh" event={"ID":"b1d3bd95-218f-44e4-9445-79df9070863f","Type":"ContainerDied","Data":"75925f9516f408b95cedbb6a0f3f4b8d31c385632b3a39dd4d4f5eeab78cee5f"}
Sep 06 10:42:18.829164 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:18.829058    2557 scope.go:117] "RemoveContainer" containerID="75925f9516f408b95cedbb6a0f3f4b8d31c385632b3a39dd4d4f5eeab78cee5f"
Sep 06 10:42:18.830154 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:18.830122    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:42:18.830211 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:18.830197    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:42:18.830269 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:18.830255    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:42:19.247630 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:19.247589    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:42:19.835627 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:19.835583    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh" event={"ID":"b1d3bd95-218f-44e4-9445-79df9070863f","Type":"ContainerStarted","Data":"e217506dc1a9c36350dcc8075b43a817b7d11d77e703e3e1504a1e4ee8939cf6"}
Sep 06 10:42:20.444680 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:20.444633    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:42:20.444935 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:20.444909    2557 patch_prober.go:28] interesting pod/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 06 10:42:20.445027 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:20.444978    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh" podUID="b1d3bd95-218f-44e4-9445-79df9070863f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 06 10:42:21.444709 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:21.444664    2557 patch_prober.go:28] interesting pod/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 06 10:42:21.445110 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:21.444730    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh" podUID="b1d3bd95-218f-44e4-9445-79df9070863f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 06 10:42:22.445202 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:22.445162    2557 patch_prober.go:28] interesting pod/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 06 10:42:22.445640 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:22.445223    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh" podUID="b1d3bd95-218f-44e4-9445-79df9070863f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 06 10:42:23.079916 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:23.079875    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2dae43f5-zqwjc"]
Sep 06 10:42:23.080774 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:23.080749    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdf97238-9455-41d4-826b-57cd25e7d39e" containerName="istio-proxy"
Sep 06 10:42:23.080924 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:23.080912    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdf97238-9455-41d4-826b-57cd25e7d39e" containerName="istio-proxy"
Sep 06 10:42:23.081041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:23.081031    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdf97238-9455-41d4-826b-57cd25e7d39e" containerName="istio-proxy"
Sep 06 10:42:23.081118 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:23.081109    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdf97238-9455-41d4-826b-57cd25e7d39e" containerName="istio-proxy"
Sep 06 10:42:23.081315 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:23.081302    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdf97238-9455-41d4-826b-57cd25e7d39e" containerName="istio-proxy"
Sep 06 10:42:23.081410 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:23.081399    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdf97238-9455-41d4-826b-57cd25e7d39e" containerName="istio-proxy"
Sep 06 10:42:23.084460 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:23.084442    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2dae43f5-zqwjc"
Sep 06 10:42:23.088747 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:23.088722    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\""
Sep 06 10:42:23.089051 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:23.088830    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 10:42:23.093625 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:23.093602    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2dae43f5-zqwjc"]
Sep 06 10:42:23.157437 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:23.157398    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71cd3d1e-4334-4a99-88f8-d3c54090df87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2dae43f5-zqwjc\" (UID: \"71cd3d1e-4334-4a99-88f8-d3c54090df87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2dae43f5-zqwjc"
Sep 06 10:42:23.258742 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:23.258708    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71cd3d1e-4334-4a99-88f8-d3c54090df87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2dae43f5-zqwjc\" (UID: \"71cd3d1e-4334-4a99-88f8-d3c54090df87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2dae43f5-zqwjc"
Sep 06 10:42:23.259336 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:23.259316    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71cd3d1e-4334-4a99-88f8-d3c54090df87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2dae43f5-zqwjc\" (UID: \"71cd3d1e-4334-4a99-88f8-d3c54090df87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2dae43f5-zqwjc"
Sep 06 10:42:23.397932 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:23.397893    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2dae43f5-zqwjc"
Sep 06 10:42:23.444591 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:23.444525    2557 patch_prober.go:28] interesting pod/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 06 10:42:23.444748 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:23.444605    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh" podUID="b1d3bd95-218f-44e4-9445-79df9070863f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 06 10:42:23.672567 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:23.670137    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2dae43f5-zqwjc"]
Sep 06 10:42:23.858941 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:23.858902    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2dae43f5-zqwjc" event={"ID":"71cd3d1e-4334-4a99-88f8-d3c54090df87","Type":"ContainerStarted","Data":"164ee7c559a7bc8876e148be1d42d2846060946a0703d09fa5b14d070a445b48"}
Sep 06 10:42:24.445239 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:24.445198    2557 patch_prober.go:28] interesting pod/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 06 10:42:24.445414 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:24.445263    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh" podUID="b1d3bd95-218f-44e4-9445-79df9070863f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 06 10:42:24.867093 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:24.867040    2557 generic.go:356] "Generic (PLEG): container finished" podID="71cd3d1e-4334-4a99-88f8-d3c54090df87" containerID="400b3191a7d7faf0c019ec529640a94d6b0eb8924e16f5880a2326735e144f9a" exitCode=0
Sep 06 10:42:24.867543 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:24.867190    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2dae43f5-zqwjc" event={"ID":"71cd3d1e-4334-4a99-88f8-d3c54090df87","Type":"ContainerDied","Data":"400b3191a7d7faf0c019ec529640a94d6b0eb8924e16f5880a2326735e144f9a"}
Sep 06 10:42:25.448672 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:25.448593    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:42:25.448926 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:25.448908    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:42:25.449478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:25.449459    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:42:26.012424 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:26.012400    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2dae43f5-zqwjc"
Sep 06 10:42:26.108945 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:26.108908    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71cd3d1e-4334-4a99-88f8-d3c54090df87-maas-api-service-ca\") pod \"71cd3d1e-4334-4a99-88f8-d3c54090df87\" (UID: \"71cd3d1e-4334-4a99-88f8-d3c54090df87\") "
Sep 06 10:42:26.109300 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:26.109279    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71cd3d1e-4334-4a99-88f8-d3c54090df87-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "71cd3d1e-4334-4a99-88f8-d3c54090df87" (UID: "71cd3d1e-4334-4a99-88f8-d3c54090df87"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:42:26.210466 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:26.210380    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71cd3d1e-4334-4a99-88f8-d3c54090df87-maas-api-service-ca\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:42:26.878373 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:26.878334    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2dae43f5-zqwjc" event={"ID":"71cd3d1e-4334-4a99-88f8-d3c54090df87","Type":"ContainerDied","Data":"164ee7c559a7bc8876e148be1d42d2846060946a0703d09fa5b14d070a445b48"}
Sep 06 10:42:26.878373 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:26.878358    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2dae43f5-zqwjc"
Sep 06 10:42:26.878667 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:26.878398    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="164ee7c559a7bc8876e148be1d42d2846060946a0703d09fa5b14d070a445b48"
Sep 06 10:42:29.286510 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:29.286454    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2dae43f5-zqwjc"]
Sep 06 10:42:29.288875 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:29.288856    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2dae43f5-zqwjc"]
Sep 06 10:42:29.421709 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:29.421672    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"]
Sep 06 10:42:29.421968 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:29.421913    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh" podUID="b1d3bd95-218f-44e4-9445-79df9070863f" containerName="istio-proxy" containerID="cri-o://e217506dc1a9c36350dcc8075b43a817b7d11d77e703e3e1504a1e4ee8939cf6" gracePeriod=30
Sep 06 10:42:30.037635 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.037600    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71cd3d1e-4334-4a99-88f8-d3c54090df87" path="/var/lib/kubelet/pods/71cd3d1e-4334-4a99-88f8-d3c54090df87/volumes"
Sep 06 10:42:30.127339 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.127301    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"]
Sep 06 10:42:30.127801 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.127782    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71cd3d1e-4334-4a99-88f8-d3c54090df87" containerName="revoke-keys"
Sep 06 10:42:30.127801 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.127803    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="71cd3d1e-4334-4a99-88f8-d3c54090df87" containerName="revoke-keys"
Sep 06 10:42:30.127968 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.127955    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="71cd3d1e-4334-4a99-88f8-d3c54090df87" containerName="revoke-keys"
Sep 06 10:42:30.130345 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.130326    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.132678 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.132656    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-2b45b43b-openshift-default-dockercfg-fs4fg\""
Sep 06 10:42:30.142797 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.142773    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"]
Sep 06 10:42:30.250754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.250713    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-workload-socket\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.250923 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.250815    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zvdv\" (UniqueName: \"kubernetes.io/projected/adf053c9-fb70-40a0-8e93-38179b59fd68-kube-api-access-4zvdv\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.250923 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.250892    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adf053c9-fb70-40a0-8e93-38179b59fd68-istiod-ca-cert\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.250923 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.250919    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-token\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.251073 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.250961    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-workload-certs\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.251073 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.251050    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-envoy\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.251142 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.251111    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-data\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.251180 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.251138    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-podinfo\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.251232 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.251176    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-credential-socket\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.353374 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.352292    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-data\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.353374 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.352352    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-podinfo\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.353374 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.352404    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-credential-socket\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.353374 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.352443    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-workload-socket\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.353374 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.352506    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zvdv\" (UniqueName: \"kubernetes.io/projected/adf053c9-fb70-40a0-8e93-38179b59fd68-kube-api-access-4zvdv\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.353374 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.352572    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adf053c9-fb70-40a0-8e93-38179b59fd68-istiod-ca-cert\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.353374 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.352606    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-token\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.353374 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.352669    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-workload-certs\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.353374 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.352733    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-envoy\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.358405 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.355705    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-envoy\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.358405 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.355969    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-data\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.358838 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.358588    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-podinfo\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.359157 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.358932    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-credential-socket\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.359459 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.359161    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-workload-socket\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.359459 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.359252    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-workload-certs\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.359958 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.359939    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adf053c9-fb70-40a0-8e93-38179b59fd68-istiod-ca-cert\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.371827 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.371807    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-token\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.371969 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.371952    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zvdv\" (UniqueName: \"kubernetes.io/projected/adf053c9-fb70-40a0-8e93-38179b59fd68-kube-api-access-4zvdv\") pod \"e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") " pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.443008 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.442982    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:30.719174 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.719141    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"]
Sep 06 10:42:30.720551 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:42:30.720522    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadf053c9_fb70_40a0_8e93_38179b59fd68.slice/crio-966b9d3fdfb28a50ac1e8e284a5c2d38bfcb3836b7305537d7d9afa866504492 WatchSource:0}: Error finding container 966b9d3fdfb28a50ac1e8e284a5c2d38bfcb3836b7305537d7d9afa866504492: Status 404 returned error can't find the container with id 966b9d3fdfb28a50ac1e8e284a5c2d38bfcb3836b7305537d7d9afa866504492
Sep 06 10:42:30.722918 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.722882    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:42:30.722987 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.722963    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:42:30.723023 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.722997    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:42:30.901799 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.901767    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt" event={"ID":"adf053c9-fb70-40a0-8e93-38179b59fd68","Type":"ContainerStarted","Data":"6384d021f96f312d7dd5f8065b40281ab8e216d0457afd01f58e8577c47ab103"}
Sep 06 10:42:30.902002 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.901819    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt" event={"ID":"adf053c9-fb70-40a0-8e93-38179b59fd68","Type":"ContainerStarted","Data":"966b9d3fdfb28a50ac1e8e284a5c2d38bfcb3836b7305537d7d9afa866504492"}
Sep 06 10:42:30.921722 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:30.921666    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt" podStartSLOduration=0.921650448 podStartE2EDuration="921.650448ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:42:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:42:30.920524169 +0000 UTC m=+763.370304938" watchObservedRunningTime="2026-09-06 10:42:30.921650448 +0000 UTC m=+763.371431245"
Sep 06 10:42:31.443935 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:31.443903    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:31.448710 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:31.448684    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:31.906910 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:31.906881    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:31.907835 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:31.907816    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:34.675289 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.675264    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:42:34.690711 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.690554    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-workload-certs\") pod \"b1d3bd95-218f-44e4-9445-79df9070863f\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") "
Sep 06 10:42:34.690711 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.690684    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1d3bd95-218f-44e4-9445-79df9070863f-istiod-ca-cert\") pod \"b1d3bd95-218f-44e4-9445-79df9070863f\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") "
Sep 06 10:42:34.690939 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.690715    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-workload-socket\") pod \"b1d3bd95-218f-44e4-9445-79df9070863f\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") "
Sep 06 10:42:34.690939 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.690749    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-istio-envoy\") pod \"b1d3bd95-218f-44e4-9445-79df9070863f\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") "
Sep 06 10:42:34.690939 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.690775    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-credential-socket\") pod \"b1d3bd95-218f-44e4-9445-79df9070863f\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") "
Sep 06 10:42:34.690939 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.690830    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b1d3bd95-218f-44e4-9445-79df9070863f" (UID: "b1d3bd95-218f-44e4-9445-79df9070863f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:42:34.690939 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.690854    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1d3bd95-218f-44e4-9445-79df9070863f-istio-token\") pod \"b1d3bd95-218f-44e4-9445-79df9070863f\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") "
Sep 06 10:42:34.690939 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.690894    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1d3bd95-218f-44e4-9445-79df9070863f-istio-podinfo\") pod \"b1d3bd95-218f-44e4-9445-79df9070863f\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") "
Sep 06 10:42:34.691256 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.690957    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-istio-data\") pod \"b1d3bd95-218f-44e4-9445-79df9070863f\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") "
Sep 06 10:42:34.691256 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.691002    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rwjj\" (UniqueName: \"kubernetes.io/projected/b1d3bd95-218f-44e4-9445-79df9070863f-kube-api-access-5rwjj\") pod \"b1d3bd95-218f-44e4-9445-79df9070863f\" (UID: \"b1d3bd95-218f-44e4-9445-79df9070863f\") "
Sep 06 10:42:34.691256 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.691021    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b1d3bd95-218f-44e4-9445-79df9070863f" (UID: "b1d3bd95-218f-44e4-9445-79df9070863f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:42:34.691256 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.691059    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b1d3bd95-218f-44e4-9445-79df9070863f" (UID: "b1d3bd95-218f-44e4-9445-79df9070863f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:42:34.691256 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.691070    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1d3bd95-218f-44e4-9445-79df9070863f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b1d3bd95-218f-44e4-9445-79df9070863f" (UID: "b1d3bd95-218f-44e4-9445-79df9070863f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:42:34.691477 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.691270    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-istio-data" (OuterVolumeSpecName: "istio-data") pod "b1d3bd95-218f-44e4-9445-79df9070863f" (UID: "b1d3bd95-218f-44e4-9445-79df9070863f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:42:34.691477 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.691374    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-istio-data\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:42:34.691477 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.691391    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-workload-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:42:34.691477 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.691404    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1d3bd95-218f-44e4-9445-79df9070863f-istiod-ca-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:42:34.691477 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.691416    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-workload-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:42:34.691477 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.691429    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-credential-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:42:34.693090 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.693038    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b1d3bd95-218f-44e4-9445-79df9070863f" (UID: "b1d3bd95-218f-44e4-9445-79df9070863f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:42:34.693279 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.693237    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1d3bd95-218f-44e4-9445-79df9070863f-kube-api-access-5rwjj" (OuterVolumeSpecName: "kube-api-access-5rwjj") pod "b1d3bd95-218f-44e4-9445-79df9070863f" (UID: "b1d3bd95-218f-44e4-9445-79df9070863f"). InnerVolumeSpecName "kube-api-access-5rwjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:42:34.693389 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.693329    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1d3bd95-218f-44e4-9445-79df9070863f-istio-token" (OuterVolumeSpecName: "istio-token") pod "b1d3bd95-218f-44e4-9445-79df9070863f" (UID: "b1d3bd95-218f-44e4-9445-79df9070863f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:42:34.693460 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.693416    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b1d3bd95-218f-44e4-9445-79df9070863f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b1d3bd95-218f-44e4-9445-79df9070863f" (UID: "b1d3bd95-218f-44e4-9445-79df9070863f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 10:42:34.792921 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.792891    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rwjj\" (UniqueName: \"kubernetes.io/projected/b1d3bd95-218f-44e4-9445-79df9070863f-kube-api-access-5rwjj\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:42:34.792921 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.792918    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1d3bd95-218f-44e4-9445-79df9070863f-istio-envoy\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:42:34.792921 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.792930    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1d3bd95-218f-44e4-9445-79df9070863f-istio-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:42:34.793155 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.792939    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1d3bd95-218f-44e4-9445-79df9070863f-istio-podinfo\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:42:34.924841 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.924805    2557 generic.go:356] "Generic (PLEG): container finished" podID="b1d3bd95-218f-44e4-9445-79df9070863f" containerID="e217506dc1a9c36350dcc8075b43a817b7d11d77e703e3e1504a1e4ee8939cf6" exitCode=0
Sep 06 10:42:34.924986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.924866    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"
Sep 06 10:42:34.924986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.924890    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh" event={"ID":"b1d3bd95-218f-44e4-9445-79df9070863f","Type":"ContainerDied","Data":"e217506dc1a9c36350dcc8075b43a817b7d11d77e703e3e1504a1e4ee8939cf6"}
Sep 06 10:42:34.924986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.924947    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh" event={"ID":"b1d3bd95-218f-44e4-9445-79df9070863f","Type":"ContainerDied","Data":"5aa93c4599c6df146564b65f8330b423522d6acdd3b0847ad76688c41ad6d303"}
Sep 06 10:42:34.924986 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.924969    2557 scope.go:117] "RemoveContainer" containerID="e217506dc1a9c36350dcc8075b43a817b7d11d77e703e3e1504a1e4ee8939cf6"
Sep 06 10:42:34.940940 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.940923    2557 scope.go:117] "RemoveContainer" containerID="75925f9516f408b95cedbb6a0f3f4b8d31c385632b3a39dd4d4f5eeab78cee5f"
Sep 06 10:42:34.948780 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.948758    2557 scope.go:117] "RemoveContainer" containerID="e217506dc1a9c36350dcc8075b43a817b7d11d77e703e3e1504a1e4ee8939cf6"
Sep 06 10:42:34.949031 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:42:34.949012    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e217506dc1a9c36350dcc8075b43a817b7d11d77e703e3e1504a1e4ee8939cf6\": container with ID starting with e217506dc1a9c36350dcc8075b43a817b7d11d77e703e3e1504a1e4ee8939cf6 not found: ID does not exist" containerID="e217506dc1a9c36350dcc8075b43a817b7d11d77e703e3e1504a1e4ee8939cf6"
Sep 06 10:42:34.949116 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.949044    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e217506dc1a9c36350dcc8075b43a817b7d11d77e703e3e1504a1e4ee8939cf6"} err="failed to get container status \"e217506dc1a9c36350dcc8075b43a817b7d11d77e703e3e1504a1e4ee8939cf6\": rpc error: code = NotFound desc = could not find container \"e217506dc1a9c36350dcc8075b43a817b7d11d77e703e3e1504a1e4ee8939cf6\": container with ID starting with e217506dc1a9c36350dcc8075b43a817b7d11d77e703e3e1504a1e4ee8939cf6 not found: ID does not exist"
Sep 06 10:42:34.949116 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.949067    2557 scope.go:117] "RemoveContainer" containerID="75925f9516f408b95cedbb6a0f3f4b8d31c385632b3a39dd4d4f5eeab78cee5f"
Sep 06 10:42:34.949477 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:42:34.949452    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75925f9516f408b95cedbb6a0f3f4b8d31c385632b3a39dd4d4f5eeab78cee5f\": container with ID starting with 75925f9516f408b95cedbb6a0f3f4b8d31c385632b3a39dd4d4f5eeab78cee5f not found: ID does not exist" containerID="75925f9516f408b95cedbb6a0f3f4b8d31c385632b3a39dd4d4f5eeab78cee5f"
Sep 06 10:42:34.949630 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.949511    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75925f9516f408b95cedbb6a0f3f4b8d31c385632b3a39dd4d4f5eeab78cee5f"} err="failed to get container status \"75925f9516f408b95cedbb6a0f3f4b8d31c385632b3a39dd4d4f5eeab78cee5f\": rpc error: code = NotFound desc = could not find container \"75925f9516f408b95cedbb6a0f3f4b8d31c385632b3a39dd4d4f5eeab78cee5f\": container with ID starting with 75925f9516f408b95cedbb6a0f3f4b8d31c385632b3a39dd4d4f5eeab78cee5f not found: ID does not exist"
Sep 06 10:42:34.951672 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.951652    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"]
Sep 06 10:42:34.955452 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:34.955432    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-2dae43f5-gw-openshift-default-8479886f76-k46dh"]
Sep 06 10:42:36.037227 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:36.037190    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1d3bd95-218f-44e4-9445-79df9070863f" path="/var/lib/kubelet/pods/b1d3bd95-218f-44e4-9445-79df9070863f/volumes"
Sep 06 10:42:37.017815 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:37.017777    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:42:47.286600 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:47.286565    2557 patch_prober.go:28] interesting pod/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": read tcp 10.132.0.2:53368->10.132.0.51:15021: read: connection reset by peer" start-of-body=
Sep 06 10:42:47.286963 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:47.286642    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt" podUID="adf053c9-fb70-40a0-8e93-38179b59fd68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": read tcp 10.132.0.2:53368->10.132.0.51:15021: read: connection reset by peer"
Sep 06 10:42:48.010599 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:48.010564    2557 generic.go:356] "Generic (PLEG): container finished" podID="adf053c9-fb70-40a0-8e93-38179b59fd68" containerID="6384d021f96f312d7dd5f8065b40281ab8e216d0457afd01f58e8577c47ab103" exitCode=0
Sep 06 10:42:48.010768 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:48.010630    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt" event={"ID":"adf053c9-fb70-40a0-8e93-38179b59fd68","Type":"ContainerDied","Data":"6384d021f96f312d7dd5f8065b40281ab8e216d0457afd01f58e8577c47ab103"}
Sep 06 10:42:48.011060 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:48.011041    2557 scope.go:117] "RemoveContainer" containerID="6384d021f96f312d7dd5f8065b40281ab8e216d0457afd01f58e8577c47ab103"
Sep 06 10:42:48.012069 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:48.012044    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:42:48.012128 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:48.012106    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:42:48.012166 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:48.012135    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:42:49.021253 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:49.021218    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt" event={"ID":"adf053c9-fb70-40a0-8e93-38179b59fd68","Type":"ContainerStarted","Data":"79ac409c3f2cbb47096f7af604b49a8a158c8620f67adc655c11f2b86ab1f35b"}
Sep 06 10:42:49.443473 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:49.443443    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:49.443682 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:49.443638    2557 patch_prober.go:28] interesting pod/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 06 10:42:49.443735 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:49.443689    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt" podUID="adf053c9-fb70-40a0-8e93-38179b59fd68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 06 10:42:50.443376 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:50.443338    2557 patch_prober.go:28] interesting pod/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 06 10:42:50.443780 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:50.443410    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt" podUID="adf053c9-fb70-40a0-8e93-38179b59fd68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 06 10:42:51.443743 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:51.443704    2557 patch_prober.go:28] interesting pod/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 06 10:42:51.444143 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:51.443780    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt" podUID="adf053c9-fb70-40a0-8e93-38179b59fd68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 06 10:42:52.443750 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:52.443710    2557 patch_prober.go:28] interesting pod/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 06 10:42:52.444157 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:52.443782    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt" podUID="adf053c9-fb70-40a0-8e93-38179b59fd68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 06 10:42:53.443709 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:53.443670    2557 patch_prober.go:28] interesting pod/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 06 10:42:53.443887 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:53.443727    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt" podUID="adf053c9-fb70-40a0-8e93-38179b59fd68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 06 10:42:54.447695 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:54.447663    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:54.448106 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:54.448066    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:54.448617 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:54.448598    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:42:55.212550 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:55.212518    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:42:57.959442 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:57.959403    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c"]
Sep 06 10:42:57.960282 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:57.960257    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1d3bd95-218f-44e4-9445-79df9070863f" containerName="istio-proxy"
Sep 06 10:42:57.960282 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:57.960284    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1d3bd95-218f-44e4-9445-79df9070863f" containerName="istio-proxy"
Sep 06 10:42:57.960441 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:57.960305    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1d3bd95-218f-44e4-9445-79df9070863f" containerName="istio-proxy"
Sep 06 10:42:57.960441 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:57.960313    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1d3bd95-218f-44e4-9445-79df9070863f" containerName="istio-proxy"
Sep 06 10:42:57.960587 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:57.960572    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1d3bd95-218f-44e4-9445-79df9070863f" containerName="istio-proxy"
Sep 06 10:42:57.964778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:57.964756    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c"
Sep 06 10:42:57.966562 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:57.966537    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c"]
Sep 06 10:42:57.967239 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:57.967217    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\""
Sep 06 10:42:57.967350 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:57.967222    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 10:42:58.118854 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:58.118824    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ee8da1-a008-437b-be27-bc70f103f60f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c\" (UID: \"16ee8da1-a008-437b-be27-bc70f103f60f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c"
Sep 06 10:42:58.219442 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:58.219352    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ee8da1-a008-437b-be27-bc70f103f60f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c\" (UID: \"16ee8da1-a008-437b-be27-bc70f103f60f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c"
Sep 06 10:42:58.220018 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:58.219995    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ee8da1-a008-437b-be27-bc70f103f60f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c\" (UID: \"16ee8da1-a008-437b-be27-bc70f103f60f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c"
Sep 06 10:42:58.278500 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:58.278473    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c"
Sep 06 10:42:58.550178 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:58.550149    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c"]
Sep 06 10:42:58.552216 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:42:58.552193    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16ee8da1_a008_437b_be27_bc70f103f60f.slice/crio-8c10ae041e8233063c14df51de4034690743266b61559f3da98575eb63d2b548 WatchSource:0}: Error finding container 8c10ae041e8233063c14df51de4034690743266b61559f3da98575eb63d2b548: Status 404 returned error can't find the container with id 8c10ae041e8233063c14df51de4034690743266b61559f3da98575eb63d2b548
Sep 06 10:42:59.073306 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:59.073272    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c" event={"ID":"16ee8da1-a008-437b-be27-bc70f103f60f","Type":"ContainerStarted","Data":"372aed3c43494df98f80c682122091b536e6734ed94154f4c28123daff288027"}
Sep 06 10:42:59.073741 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:59.073326    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c" event={"ID":"16ee8da1-a008-437b-be27-bc70f103f60f","Type":"ContainerStarted","Data":"8c10ae041e8233063c14df51de4034690743266b61559f3da98575eb63d2b548"}
Sep 06 10:42:59.088291 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:42:59.088243    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c" podStartSLOduration=2.08822761 podStartE2EDuration="2.08822761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:42:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:42:59.08641587 +0000 UTC m=+791.536196637" watchObservedRunningTime="2026-09-06 10:42:59.08822761 +0000 UTC m=+791.538008379"
Sep 06 10:43:00.080283 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:00.080246    2557 generic.go:356] "Generic (PLEG): container finished" podID="16ee8da1-a008-437b-be27-bc70f103f60f" containerID="372aed3c43494df98f80c682122091b536e6734ed94154f4c28123daff288027" exitCode=0
Sep 06 10:43:00.080696 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:00.080287    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c" event={"ID":"16ee8da1-a008-437b-be27-bc70f103f60f","Type":"ContainerDied","Data":"372aed3c43494df98f80c682122091b536e6734ed94154f4c28123daff288027"}
Sep 06 10:43:01.237343 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:01.237321    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c"
Sep 06 10:43:01.349785 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:01.349759    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ee8da1-a008-437b-be27-bc70f103f60f-maas-api-service-ca\") pod \"16ee8da1-a008-437b-be27-bc70f103f60f\" (UID: \"16ee8da1-a008-437b-be27-bc70f103f60f\") "
Sep 06 10:43:01.350097 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:01.350075    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16ee8da1-a008-437b-be27-bc70f103f60f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "16ee8da1-a008-437b-be27-bc70f103f60f" (UID: "16ee8da1-a008-437b-be27-bc70f103f60f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:43:01.450791 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:01.450765    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ee8da1-a008-437b-be27-bc70f103f60f-maas-api-service-ca\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:02.092051 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:02.092014    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c" event={"ID":"16ee8da1-a008-437b-be27-bc70f103f60f","Type":"ContainerDied","Data":"8c10ae041e8233063c14df51de4034690743266b61559f3da98575eb63d2b548"}
Sep 06 10:43:02.092200 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:02.092062    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c10ae041e8233063c14df51de4034690743266b61559f3da98575eb63d2b548"
Sep 06 10:43:02.092200 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:02.092028    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c"
Sep 06 10:43:04.525795 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:04.525761    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c"]
Sep 06 10:43:04.528772 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:04.528743    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b45b43b-wf25c"]
Sep 06 10:43:05.613499 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:05.613455    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"]
Sep 06 10:43:05.613867 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:05.613739    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt" podUID="adf053c9-fb70-40a0-8e93-38179b59fd68" containerName="istio-proxy" containerID="cri-o://79ac409c3f2cbb47096f7af604b49a8a158c8620f67adc655c11f2b86ab1f35b" gracePeriod=30
Sep 06 10:43:06.037931 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.037857    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16ee8da1-a008-437b-be27-bc70f103f60f" path="/var/lib/kubelet/pods/16ee8da1-a008-437b-be27-bc70f103f60f/volumes"
Sep 06 10:43:06.298980 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.298898    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"]
Sep 06 10:43:06.299528 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.299507    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16ee8da1-a008-437b-be27-bc70f103f60f" containerName="revoke-keys"
Sep 06 10:43:06.299608 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.299531    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="16ee8da1-a008-437b-be27-bc70f103f60f" containerName="revoke-keys"
Sep 06 10:43:06.299713 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.299699    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1d3bd95-218f-44e4-9445-79df9070863f" containerName="istio-proxy"
Sep 06 10:43:06.299770 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.299719    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="16ee8da1-a008-437b-be27-bc70f103f60f" containerName="revoke-keys"
Sep 06 10:43:06.303302 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.303280    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.305754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.305674    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-02ec029f-openshift-default-dockercfg-q4hdd\""
Sep 06 10:43:06.309271 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.309247    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"]
Sep 06 10:43:06.399134 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.399100    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-workload-certs\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.399317 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.399153    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-istio-envoy\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.399317 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.399178    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84520080-e92f-4385-8165-9689c7a7ebd8-istio-token\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.399402 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.399306    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84520080-e92f-4385-8165-9689c7a7ebd8-istiod-ca-cert\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.399402 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.399379    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-workload-socket\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.399472 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.399450    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-istio-data\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.399536 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.399483    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl6vr\" (UniqueName: \"kubernetes.io/projected/84520080-e92f-4385-8165-9689c7a7ebd8-kube-api-access-bl6vr\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.399597 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.399566    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-credential-socket\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.399643 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.399630    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84520080-e92f-4385-8165-9689c7a7ebd8-istio-podinfo\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.500621 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.500577    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-workload-socket\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.500787 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.500736    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-istio-data\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.500787 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.500776    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl6vr\" (UniqueName: \"kubernetes.io/projected/84520080-e92f-4385-8165-9689c7a7ebd8-kube-api-access-bl6vr\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.500865 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.500821    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-credential-socket\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.500865 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.500851    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84520080-e92f-4385-8165-9689c7a7ebd8-istio-podinfo\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.500951 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.500880    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-workload-certs\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.500951 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.500921    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-istio-envoy\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.501060 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.500946    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84520080-e92f-4385-8165-9689c7a7ebd8-istio-token\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.501060 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.500986    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-workload-socket\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.501060 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.501015    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84520080-e92f-4385-8165-9689c7a7ebd8-istiod-ca-cert\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.501222 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.501067    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-istio-data\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.501222 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.501170    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-credential-socket\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.501326 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.501305    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-workload-certs\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.501839 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.501813    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84520080-e92f-4385-8165-9689c7a7ebd8-istiod-ca-cert\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.503475 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.503447    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84520080-e92f-4385-8165-9689c7a7ebd8-istio-podinfo\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.503630 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.503561    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-istio-envoy\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.508666 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.508643    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84520080-e92f-4385-8165-9689c7a7ebd8-istio-token\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.508666 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.508652    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl6vr\" (UniqueName: \"kubernetes.io/projected/84520080-e92f-4385-8165-9689c7a7ebd8-kube-api-access-bl6vr\") pod \"e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") " pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.616628 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.616601    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:06.892934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.892907    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"]
Sep 06 10:43:06.894967 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:43:06.894935    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84520080_e92f_4385_8165_9689c7a7ebd8.slice/crio-8da9e3454c010f8e92c93f9dac40d3ea75618e35bb1c9b294fcfe26bbe0fb13e WatchSource:0}: Error finding container 8da9e3454c010f8e92c93f9dac40d3ea75618e35bb1c9b294fcfe26bbe0fb13e: Status 404 returned error can't find the container with id 8da9e3454c010f8e92c93f9dac40d3ea75618e35bb1c9b294fcfe26bbe0fb13e
Sep 06 10:43:06.897530 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.897469    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:43:06.897620 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.897551    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:43:06.897620 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:06.897598    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:43:07.122672 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:07.122629    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f" event={"ID":"84520080-e92f-4385-8165-9689c7a7ebd8","Type":"ContainerStarted","Data":"4a2db8efc82c9d99547b7fe7afe77cae9109cedc73f61fc4d9a3b9790bda25e5"}
Sep 06 10:43:07.122858 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:07.122695    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f" event={"ID":"84520080-e92f-4385-8165-9689c7a7ebd8","Type":"ContainerStarted","Data":"8da9e3454c010f8e92c93f9dac40d3ea75618e35bb1c9b294fcfe26bbe0fb13e"}
Sep 06 10:43:07.141102 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:07.141060    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f" podStartSLOduration=1.141044082 podStartE2EDuration="1.141044082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:43:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:43:07.139501688 +0000 UTC m=+799.589282449" watchObservedRunningTime="2026-09-06 10:43:07.141044082 +0000 UTC m=+799.590824850"
Sep 06 10:43:07.616892 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:07.616853    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:07.621616 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:07.621592    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:08.127768 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:08.127738    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:08.128736 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:08.128720    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:09.450396 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:09.450357    2557 patch_prober.go:28] interesting pod/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 06 10:43:09.450785 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:09.450416    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt" podUID="adf053c9-fb70-40a0-8e93-38179b59fd68" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 10:43:10.864122 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:10.864094    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:43:11.043802 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.043726    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zvdv\" (UniqueName: \"kubernetes.io/projected/adf053c9-fb70-40a0-8e93-38179b59fd68-kube-api-access-4zvdv\") pod \"adf053c9-fb70-40a0-8e93-38179b59fd68\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") "
Sep 06 10:43:11.043802 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.043761    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-credential-socket\") pod \"adf053c9-fb70-40a0-8e93-38179b59fd68\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") "
Sep 06 10:43:11.043802 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.043790    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-token\") pod \"adf053c9-fb70-40a0-8e93-38179b59fd68\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") "
Sep 06 10:43:11.044031 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.043808    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-workload-socket\") pod \"adf053c9-fb70-40a0-8e93-38179b59fd68\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") "
Sep 06 10:43:11.044031 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.043843    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adf053c9-fb70-40a0-8e93-38179b59fd68-istiod-ca-cert\") pod \"adf053c9-fb70-40a0-8e93-38179b59fd68\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") "
Sep 06 10:43:11.044031 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.043876    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-envoy\") pod \"adf053c9-fb70-40a0-8e93-38179b59fd68\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") "
Sep 06 10:43:11.044455 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.044389    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "adf053c9-fb70-40a0-8e93-38179b59fd68" (UID: "adf053c9-fb70-40a0-8e93-38179b59fd68"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:43:11.044613 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.044553    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-workload-certs\") pod \"adf053c9-fb70-40a0-8e93-38179b59fd68\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") "
Sep 06 10:43:11.044613 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.044545    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "adf053c9-fb70-40a0-8e93-38179b59fd68" (UID: "adf053c9-fb70-40a0-8e93-38179b59fd68"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:43:11.044740 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.044700    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adf053c9-fb70-40a0-8e93-38179b59fd68-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "adf053c9-fb70-40a0-8e93-38179b59fd68" (UID: "adf053c9-fb70-40a0-8e93-38179b59fd68"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:43:11.046698 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.044792    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "adf053c9-fb70-40a0-8e93-38179b59fd68" (UID: "adf053c9-fb70-40a0-8e93-38179b59fd68"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:43:11.046698 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.044883    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-data\") pod \"adf053c9-fb70-40a0-8e93-38179b59fd68\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") "
Sep 06 10:43:11.046698 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.044943    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-podinfo\") pod \"adf053c9-fb70-40a0-8e93-38179b59fd68\" (UID: \"adf053c9-fb70-40a0-8e93-38179b59fd68\") "
Sep 06 10:43:11.046698 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.045514    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-credential-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:11.046698 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.045534    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-workload-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:11.046698 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.045549    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adf053c9-fb70-40a0-8e93-38179b59fd68-istiod-ca-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:11.046698 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.045563    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-workload-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:11.047595 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.047562    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "adf053c9-fb70-40a0-8e93-38179b59fd68" (UID: "adf053c9-fb70-40a0-8e93-38179b59fd68"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:43:11.047714 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.047639    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-token" (OuterVolumeSpecName: "istio-token") pod "adf053c9-fb70-40a0-8e93-38179b59fd68" (UID: "adf053c9-fb70-40a0-8e93-38179b59fd68"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:43:11.047794 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.047774    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-data" (OuterVolumeSpecName: "istio-data") pod "adf053c9-fb70-40a0-8e93-38179b59fd68" (UID: "adf053c9-fb70-40a0-8e93-38179b59fd68"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:43:11.053694 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.053672    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "adf053c9-fb70-40a0-8e93-38179b59fd68" (UID: "adf053c9-fb70-40a0-8e93-38179b59fd68"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 10:43:11.053784 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.053698    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adf053c9-fb70-40a0-8e93-38179b59fd68-kube-api-access-4zvdv" (OuterVolumeSpecName: "kube-api-access-4zvdv") pod "adf053c9-fb70-40a0-8e93-38179b59fd68" (UID: "adf053c9-fb70-40a0-8e93-38179b59fd68"). InnerVolumeSpecName "kube-api-access-4zvdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:43:11.144758 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.144731    2557 generic.go:356] "Generic (PLEG): container finished" podID="adf053c9-fb70-40a0-8e93-38179b59fd68" containerID="79ac409c3f2cbb47096f7af604b49a8a158c8620f67adc655c11f2b86ab1f35b" exitCode=0
Sep 06 10:43:11.144858 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.144791    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"
Sep 06 10:43:11.144858 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.144806    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt" event={"ID":"adf053c9-fb70-40a0-8e93-38179b59fd68","Type":"ContainerDied","Data":"79ac409c3f2cbb47096f7af604b49a8a158c8620f67adc655c11f2b86ab1f35b"}
Sep 06 10:43:11.144932 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.144877    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt" event={"ID":"adf053c9-fb70-40a0-8e93-38179b59fd68","Type":"ContainerDied","Data":"966b9d3fdfb28a50ac1e8e284a5c2d38bfcb3836b7305537d7d9afa866504492"}
Sep 06 10:43:11.144932 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.144899    2557 scope.go:117] "RemoveContainer" containerID="79ac409c3f2cbb47096f7af604b49a8a158c8620f67adc655c11f2b86ab1f35b"
Sep 06 10:43:11.145993 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.145939    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zvdv\" (UniqueName: \"kubernetes.io/projected/adf053c9-fb70-40a0-8e93-38179b59fd68-kube-api-access-4zvdv\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:11.145993 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.145957    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:11.145993 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.145966    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-envoy\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:11.145993 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.145974    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-data\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:11.145993 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.145981    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adf053c9-fb70-40a0-8e93-38179b59fd68-istio-podinfo\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:11.153934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.153918    2557 scope.go:117] "RemoveContainer" containerID="6384d021f96f312d7dd5f8065b40281ab8e216d0457afd01f58e8577c47ab103"
Sep 06 10:43:11.161939 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.161921    2557 scope.go:117] "RemoveContainer" containerID="79ac409c3f2cbb47096f7af604b49a8a158c8620f67adc655c11f2b86ab1f35b"
Sep 06 10:43:11.162199 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:43:11.162180    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79ac409c3f2cbb47096f7af604b49a8a158c8620f67adc655c11f2b86ab1f35b\": container with ID starting with 79ac409c3f2cbb47096f7af604b49a8a158c8620f67adc655c11f2b86ab1f35b not found: ID does not exist" containerID="79ac409c3f2cbb47096f7af604b49a8a158c8620f67adc655c11f2b86ab1f35b"
Sep 06 10:43:11.162246 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.162206    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79ac409c3f2cbb47096f7af604b49a8a158c8620f67adc655c11f2b86ab1f35b"} err="failed to get container status \"79ac409c3f2cbb47096f7af604b49a8a158c8620f67adc655c11f2b86ab1f35b\": rpc error: code = NotFound desc = could not find container \"79ac409c3f2cbb47096f7af604b49a8a158c8620f67adc655c11f2b86ab1f35b\": container with ID starting with 79ac409c3f2cbb47096f7af604b49a8a158c8620f67adc655c11f2b86ab1f35b not found: ID does not exist"
Sep 06 10:43:11.162246 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.162223    2557 scope.go:117] "RemoveContainer" containerID="6384d021f96f312d7dd5f8065b40281ab8e216d0457afd01f58e8577c47ab103"
Sep 06 10:43:11.162479 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:43:11.162462    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6384d021f96f312d7dd5f8065b40281ab8e216d0457afd01f58e8577c47ab103\": container with ID starting with 6384d021f96f312d7dd5f8065b40281ab8e216d0457afd01f58e8577c47ab103 not found: ID does not exist" containerID="6384d021f96f312d7dd5f8065b40281ab8e216d0457afd01f58e8577c47ab103"
Sep 06 10:43:11.162571 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.162484    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6384d021f96f312d7dd5f8065b40281ab8e216d0457afd01f58e8577c47ab103"} err="failed to get container status \"6384d021f96f312d7dd5f8065b40281ab8e216d0457afd01f58e8577c47ab103\": rpc error: code = NotFound desc = could not find container \"6384d021f96f312d7dd5f8065b40281ab8e216d0457afd01f58e8577c47ab103\": container with ID starting with 6384d021f96f312d7dd5f8065b40281ab8e216d0457afd01f58e8577c47ab103 not found: ID does not exist"
Sep 06 10:43:11.166794 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.166769    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"]
Sep 06 10:43:11.169592 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.169569    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-2b45b43b-openshift-default-64775f4bd8-cm4zt"]
Sep 06 10:43:11.315785 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:11.315724    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:43:12.039644 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:12.039602    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adf053c9-fb70-40a0-8e93-38179b59fd68" path="/var/lib/kubelet/pods/adf053c9-fb70-40a0-8e93-38179b59fd68/volumes"
Sep 06 10:43:26.778929 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:26.778885    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-55111c-5h9ln"]
Sep 06 10:43:26.779560 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:26.779537    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adf053c9-fb70-40a0-8e93-38179b59fd68" containerName="istio-proxy"
Sep 06 10:43:26.779673 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:26.779563    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="adf053c9-fb70-40a0-8e93-38179b59fd68" containerName="istio-proxy"
Sep 06 10:43:26.779673 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:26.779583    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adf053c9-fb70-40a0-8e93-38179b59fd68" containerName="istio-proxy"
Sep 06 10:43:26.779673 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:26.779592    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="adf053c9-fb70-40a0-8e93-38179b59fd68" containerName="istio-proxy"
Sep 06 10:43:26.779845 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:26.779711    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="adf053c9-fb70-40a0-8e93-38179b59fd68" containerName="istio-proxy"
Sep 06 10:43:26.779845 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:26.779726    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="adf053c9-fb70-40a0-8e93-38179b59fd68" containerName="istio-proxy"
Sep 06 10:43:26.783537 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:26.783512    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-55111c-5h9ln"
Sep 06 10:43:26.788348 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:26.788315    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-55111c-5h9ln"]
Sep 06 10:43:26.788684 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:26.788661    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 10:43:26.790274 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:26.790247    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\""
Sep 06 10:43:26.902218 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:26.902182    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-55111c-5h9ln\" (UID: \"bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-55111c-5h9ln"
Sep 06 10:43:27.003686 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:27.003655    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-55111c-5h9ln\" (UID: \"bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-55111c-5h9ln"
Sep 06 10:43:27.004269 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:27.004249    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-55111c-5h9ln\" (UID: \"bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-55111c-5h9ln"
Sep 06 10:43:27.101549 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:27.101512    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-55111c-5h9ln"
Sep 06 10:43:27.372200 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:27.372106    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-55111c-5h9ln"]
Sep 06 10:43:27.378339 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:43:27.378293    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc3e3331_7d5d_4763_acfd_f1aa0e1c4f4d.slice/crio-14fc513f0db69330c0d47c53ce409b3a6826e90b4aeac75f347045d4180447fd WatchSource:0}: Error finding container 14fc513f0db69330c0d47c53ce409b3a6826e90b4aeac75f347045d4180447fd: Status 404 returned error can't find the container with id 14fc513f0db69330c0d47c53ce409b3a6826e90b4aeac75f347045d4180447fd
Sep 06 10:43:28.233095 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:28.233055    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-55111c-5h9ln" event={"ID":"bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d","Type":"ContainerStarted","Data":"7c6e82be54cec680ddd1702a46934934a4c3c7e9b3ff3a900d98dbb44715e0dc"}
Sep 06 10:43:28.233600 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:28.233124    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-55111c-5h9ln" event={"ID":"bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d","Type":"ContainerStarted","Data":"14fc513f0db69330c0d47c53ce409b3a6826e90b4aeac75f347045d4180447fd"}
Sep 06 10:43:28.247555 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:28.247461    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-55111c-5h9ln" podStartSLOduration=2.247445458 podStartE2EDuration="2.247445458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:43:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:43:28.246995661 +0000 UTC m=+820.696776429" watchObservedRunningTime="2026-09-06 10:43:28.247445458 +0000 UTC m=+820.697226210"
Sep 06 10:43:29.241877 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:29.241842    2557 generic.go:356] "Generic (PLEG): container finished" podID="bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d" containerID="7c6e82be54cec680ddd1702a46934934a4c3c7e9b3ff3a900d98dbb44715e0dc" exitCode=0
Sep 06 10:43:29.242382 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:29.241898    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-55111c-5h9ln" event={"ID":"bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d","Type":"ContainerDied","Data":"7c6e82be54cec680ddd1702a46934934a4c3c7e9b3ff3a900d98dbb44715e0dc"}
Sep 06 10:43:29.520178 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:29.520072    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:43:30.398757 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:30.398720    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-55111c-5h9ln"
Sep 06 10:43:30.540393 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:30.540358    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d-maas-api-service-ca\") pod \"bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d\" (UID: \"bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d\") "
Sep 06 10:43:30.540759 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:30.540732    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d" (UID: "bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:43:30.641974 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:30.641939    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d-maas-api-service-ca\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:31.255056 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:31.255021    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-55111c-5h9ln"
Sep 06 10:43:31.255056 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:31.255033    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-55111c-5h9ln" event={"ID":"bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d","Type":"ContainerDied","Data":"14fc513f0db69330c0d47c53ce409b3a6826e90b4aeac75f347045d4180447fd"}
Sep 06 10:43:31.255295 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:31.255079    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14fc513f0db69330c0d47c53ce409b3a6826e90b4aeac75f347045d4180447fd"
Sep 06 10:43:31.256711 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:31.256682    2557 generic.go:356] "Generic (PLEG): container finished" podID="84520080-e92f-4385-8165-9689c7a7ebd8" containerID="4a2db8efc82c9d99547b7fe7afe77cae9109cedc73f61fc4d9a3b9790bda25e5" exitCode=0
Sep 06 10:43:31.256819 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:31.256737    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f" event={"ID":"84520080-e92f-4385-8165-9689c7a7ebd8","Type":"ContainerDied","Data":"4a2db8efc82c9d99547b7fe7afe77cae9109cedc73f61fc4d9a3b9790bda25e5"}
Sep 06 10:43:31.257212 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:31.257196    2557 scope.go:117] "RemoveContainer" containerID="4a2db8efc82c9d99547b7fe7afe77cae9109cedc73f61fc4d9a3b9790bda25e5"
Sep 06 10:43:31.258280 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:31.258249    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:43:31.258386 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:31.258345    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:43:31.258449 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:31.258393    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:43:32.264564 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:32.264528    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f" event={"ID":"84520080-e92f-4385-8165-9689c7a7ebd8","Type":"ContainerStarted","Data":"594189661b69288973f3991e85d91ade2c31f61802de3d93f32c2263dcbfbfd4"}
Sep 06 10:43:32.617750 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:32.617703    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:32.618004 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:32.617981    2557 patch_prober.go:28] interesting pod/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 06 10:43:32.618089 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:32.618038    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f" podUID="84520080-e92f-4385-8165-9689c7a7ebd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 06 10:43:33.271779 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:33.271739    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk"]
Sep 06 10:43:33.286379 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:33.286349    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d" containerName="revoke-keys"
Sep 06 10:43:33.287867 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:33.287845    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d" containerName="revoke-keys"
Sep 06 10:43:33.288183 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:33.288166    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d" containerName="revoke-keys"
Sep 06 10:43:33.291925 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:33.291904    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk"]
Sep 06 10:43:33.292216 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:33.292161    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk"
Sep 06 10:43:33.295895 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:33.295872    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\""
Sep 06 10:43:33.296304 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:33.296284    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 10:43:33.382233 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:33.382177    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6551e8f-6562-462a-aebc-c43c7fb98685-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk\" (UID: \"f6551e8f-6562-462a-aebc-c43c7fb98685\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk"
Sep 06 10:43:33.491395 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:33.491337    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6551e8f-6562-462a-aebc-c43c7fb98685-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk\" (UID: \"f6551e8f-6562-462a-aebc-c43c7fb98685\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk"
Sep 06 10:43:33.492368 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:33.492325    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6551e8f-6562-462a-aebc-c43c7fb98685-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk\" (UID: \"f6551e8f-6562-462a-aebc-c43c7fb98685\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk"
Sep 06 10:43:33.604816 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:33.604772    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk"
Sep 06 10:43:33.617218 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:33.617184    2557 patch_prober.go:28] interesting pod/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 06 10:43:33.617335 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:33.617254    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f" podUID="84520080-e92f-4385-8165-9689c7a7ebd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 06 10:43:33.891243 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:33.891120    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk"]
Sep 06 10:43:33.893068 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:43:33.893003    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6551e8f_6562_462a_aebc_c43c7fb98685.slice/crio-b2ef70a40cffdfc8b265914cc855bb1e6c4d63c0e4b9f8dc775801d430cf889d WatchSource:0}: Error finding container b2ef70a40cffdfc8b265914cc855bb1e6c4d63c0e4b9f8dc775801d430cf889d: Status 404 returned error can't find the container with id b2ef70a40cffdfc8b265914cc855bb1e6c4d63c0e4b9f8dc775801d430cf889d
Sep 06 10:43:34.295433 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:34.295386    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk" event={"ID":"f6551e8f-6562-462a-aebc-c43c7fb98685","Type":"ContainerStarted","Data":"e5ee7a13cf7570dd22dfdafacf630eaa8c57cd260bd33be39e10ea409a725fe3"}
Sep 06 10:43:34.295829 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:34.295450    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk" event={"ID":"f6551e8f-6562-462a-aebc-c43c7fb98685","Type":"ContainerStarted","Data":"b2ef70a40cffdfc8b265914cc855bb1e6c4d63c0e4b9f8dc775801d430cf889d"}
Sep 06 10:43:34.310143 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:34.310083    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk" podStartSLOduration=1.31006658 podStartE2EDuration="1.31006658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:43:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:43:34.308458904 +0000 UTC m=+826.758239683" watchObservedRunningTime="2026-09-06 10:43:34.31006658 +0000 UTC m=+826.759847369"
Sep 06 10:43:34.617626 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:34.617590    2557 patch_prober.go:28] interesting pod/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 06 10:43:34.617812 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:34.617645    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f" podUID="84520080-e92f-4385-8165-9689c7a7ebd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 06 10:43:35.301891 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:35.301794    2557 generic.go:356] "Generic (PLEG): container finished" podID="f6551e8f-6562-462a-aebc-c43c7fb98685" containerID="e5ee7a13cf7570dd22dfdafacf630eaa8c57cd260bd33be39e10ea409a725fe3" exitCode=0
Sep 06 10:43:35.302299 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:35.301880    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk" event={"ID":"f6551e8f-6562-462a-aebc-c43c7fb98685","Type":"ContainerDied","Data":"e5ee7a13cf7570dd22dfdafacf630eaa8c57cd260bd33be39e10ea409a725fe3"}
Sep 06 10:43:35.617851 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:35.617810    2557 patch_prober.go:28] interesting pod/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 06 10:43:35.618033 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:35.617878    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f" podUID="84520080-e92f-4385-8165-9689c7a7ebd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 06 10:43:36.457337 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:36.457312    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk"
Sep 06 10:43:36.617216 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:36.617177    2557 patch_prober.go:28] interesting pod/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 06 10:43:36.617392 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:36.617239    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f" podUID="84520080-e92f-4385-8165-9689c7a7ebd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 06 10:43:36.624963 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:36.624939    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6551e8f-6562-462a-aebc-c43c7fb98685-maas-api-service-ca\") pod \"f6551e8f-6562-462a-aebc-c43c7fb98685\" (UID: \"f6551e8f-6562-462a-aebc-c43c7fb98685\") "
Sep 06 10:43:36.625302 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:36.625283    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6551e8f-6562-462a-aebc-c43c7fb98685-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f6551e8f-6562-462a-aebc-c43c7fb98685" (UID: "f6551e8f-6562-462a-aebc-c43c7fb98685"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:43:36.725978 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:36.725942    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6551e8f-6562-462a-aebc-c43c7fb98685-maas-api-service-ca\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:37.314205 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:37.314174    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk"
Sep 06 10:43:37.314205 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:37.314195    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk" event={"ID":"f6551e8f-6562-462a-aebc-c43c7fb98685","Type":"ContainerDied","Data":"b2ef70a40cffdfc8b265914cc855bb1e6c4d63c0e4b9f8dc775801d430cf889d"}
Sep 06 10:43:37.314437 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:37.314244    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2ef70a40cffdfc8b265914cc855bb1e6c4d63c0e4b9f8dc775801d430cf889d"
Sep 06 10:43:37.622376 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:37.622340    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:37.622854 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:37.622762    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:37.623568 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:37.623548    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:37.694414 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:37.694374    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-55111c-5h9ln"]
Sep 06 10:43:37.697066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:37.697036    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-55111c-5h9ln"]
Sep 06 10:43:38.038648 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:38.038555    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d" path="/var/lib/kubelet/pods/bc3e3331-7d5d-4763-acfd-f1aa0e1c4f4d/volumes"
Sep 06 10:43:40.710158 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:40.710124    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk"]
Sep 06 10:43:40.712522 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:40.712501    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02ec029f-6cbjk"]
Sep 06 10:43:40.834000 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:40.833967    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"]
Sep 06 10:43:40.834247 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:40.834218    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f" podUID="84520080-e92f-4385-8165-9689c7a7ebd8" containerName="istio-proxy" containerID="cri-o://594189661b69288973f3991e85d91ade2c31f61802de3d93f32c2263dcbfbfd4" gracePeriod=30
Sep 06 10:43:42.037206 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:42.037170    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6551e8f-6562-462a-aebc-c43c7fb98685" path="/var/lib/kubelet/pods/f6551e8f-6562-462a-aebc-c43c7fb98685/volumes"
Sep 06 10:43:46.100182 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.100155    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:46.109547 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.109461    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84520080-e92f-4385-8165-9689c7a7ebd8-istio-token\") pod \"84520080-e92f-4385-8165-9689c7a7ebd8\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") "
Sep 06 10:43:46.109547 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.109540    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-workload-socket\") pod \"84520080-e92f-4385-8165-9689c7a7ebd8\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") "
Sep 06 10:43:46.109750 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.109565    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-istio-envoy\") pod \"84520080-e92f-4385-8165-9689c7a7ebd8\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") "
Sep 06 10:43:46.109750 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.109591    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-workload-certs\") pod \"84520080-e92f-4385-8165-9689c7a7ebd8\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") "
Sep 06 10:43:46.109750 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.109608    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-istio-data\") pod \"84520080-e92f-4385-8165-9689c7a7ebd8\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") "
Sep 06 10:43:46.109750 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.109638    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84520080-e92f-4385-8165-9689c7a7ebd8-istiod-ca-cert\") pod \"84520080-e92f-4385-8165-9689c7a7ebd8\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") "
Sep 06 10:43:46.109750 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.109673    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-credential-socket\") pod \"84520080-e92f-4385-8165-9689c7a7ebd8\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") "
Sep 06 10:43:46.109750 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.109742    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84520080-e92f-4385-8165-9689c7a7ebd8-istio-podinfo\") pod \"84520080-e92f-4385-8165-9689c7a7ebd8\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") "
Sep 06 10:43:46.110058 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.109771    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bl6vr\" (UniqueName: \"kubernetes.io/projected/84520080-e92f-4385-8165-9689c7a7ebd8-kube-api-access-bl6vr\") pod \"84520080-e92f-4385-8165-9689c7a7ebd8\" (UID: \"84520080-e92f-4385-8165-9689c7a7ebd8\") "
Sep 06 10:43:46.110058 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.109843    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "84520080-e92f-4385-8165-9689c7a7ebd8" (UID: "84520080-e92f-4385-8165-9689c7a7ebd8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:43:46.110058 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.109911    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "84520080-e92f-4385-8165-9689c7a7ebd8" (UID: "84520080-e92f-4385-8165-9689c7a7ebd8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:43:46.110058 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.109895    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-istio-data" (OuterVolumeSpecName: "istio-data") pod "84520080-e92f-4385-8165-9689c7a7ebd8" (UID: "84520080-e92f-4385-8165-9689c7a7ebd8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:43:46.110260 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.110040    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84520080-e92f-4385-8165-9689c7a7ebd8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "84520080-e92f-4385-8165-9689c7a7ebd8" (UID: "84520080-e92f-4385-8165-9689c7a7ebd8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:43:46.110260 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.110113    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "84520080-e92f-4385-8165-9689c7a7ebd8" (UID: "84520080-e92f-4385-8165-9689c7a7ebd8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:43:46.110260 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.110148    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-workload-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:46.110260 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.110173    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-workload-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:46.110260 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.110186    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-istio-data\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:46.111728 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.111696    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84520080-e92f-4385-8165-9689c7a7ebd8-istio-token" (OuterVolumeSpecName: "istio-token") pod "84520080-e92f-4385-8165-9689c7a7ebd8" (UID: "84520080-e92f-4385-8165-9689c7a7ebd8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:43:46.111852 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.111831    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84520080-e92f-4385-8165-9689c7a7ebd8-kube-api-access-bl6vr" (OuterVolumeSpecName: "kube-api-access-bl6vr") pod "84520080-e92f-4385-8165-9689c7a7ebd8" (UID: "84520080-e92f-4385-8165-9689c7a7ebd8"). InnerVolumeSpecName "kube-api-access-bl6vr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:43:46.111937 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.111915    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "84520080-e92f-4385-8165-9689c7a7ebd8" (UID: "84520080-e92f-4385-8165-9689c7a7ebd8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:43:46.111937 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.111927    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/84520080-e92f-4385-8165-9689c7a7ebd8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "84520080-e92f-4385-8165-9689c7a7ebd8" (UID: "84520080-e92f-4385-8165-9689c7a7ebd8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 10:43:46.211203 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.211164    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-istio-envoy\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:46.211203 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.211197    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84520080-e92f-4385-8165-9689c7a7ebd8-istiod-ca-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:46.211203 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.211207    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84520080-e92f-4385-8165-9689c7a7ebd8-credential-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:46.211203 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.211216    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84520080-e92f-4385-8165-9689c7a7ebd8-istio-podinfo\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:46.211526 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.211225    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bl6vr\" (UniqueName: \"kubernetes.io/projected/84520080-e92f-4385-8165-9689c7a7ebd8-kube-api-access-bl6vr\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:46.211526 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.211234    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84520080-e92f-4385-8165-9689c7a7ebd8-istio-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:46.364607 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.364485    2557 generic.go:356] "Generic (PLEG): container finished" podID="84520080-e92f-4385-8165-9689c7a7ebd8" containerID="594189661b69288973f3991e85d91ade2c31f61802de3d93f32c2263dcbfbfd4" exitCode=0
Sep 06 10:43:46.364607 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.364545    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"
Sep 06 10:43:46.364607 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.364561    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f" event={"ID":"84520080-e92f-4385-8165-9689c7a7ebd8","Type":"ContainerDied","Data":"594189661b69288973f3991e85d91ade2c31f61802de3d93f32c2263dcbfbfd4"}
Sep 06 10:43:46.364878 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.364627    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f" event={"ID":"84520080-e92f-4385-8165-9689c7a7ebd8","Type":"ContainerDied","Data":"8da9e3454c010f8e92c93f9dac40d3ea75618e35bb1c9b294fcfe26bbe0fb13e"}
Sep 06 10:43:46.364878 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.364656    2557 scope.go:117] "RemoveContainer" containerID="594189661b69288973f3991e85d91ade2c31f61802de3d93f32c2263dcbfbfd4"
Sep 06 10:43:46.374889 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.374868    2557 scope.go:117] "RemoveContainer" containerID="4a2db8efc82c9d99547b7fe7afe77cae9109cedc73f61fc4d9a3b9790bda25e5"
Sep 06 10:43:46.383725 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.383705    2557 scope.go:117] "RemoveContainer" containerID="594189661b69288973f3991e85d91ade2c31f61802de3d93f32c2263dcbfbfd4"
Sep 06 10:43:46.384104 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:43:46.384067    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"594189661b69288973f3991e85d91ade2c31f61802de3d93f32c2263dcbfbfd4\": container with ID starting with 594189661b69288973f3991e85d91ade2c31f61802de3d93f32c2263dcbfbfd4 not found: ID does not exist" containerID="594189661b69288973f3991e85d91ade2c31f61802de3d93f32c2263dcbfbfd4"
Sep 06 10:43:46.384212 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.384115    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"594189661b69288973f3991e85d91ade2c31f61802de3d93f32c2263dcbfbfd4"} err="failed to get container status \"594189661b69288973f3991e85d91ade2c31f61802de3d93f32c2263dcbfbfd4\": rpc error: code = NotFound desc = could not find container \"594189661b69288973f3991e85d91ade2c31f61802de3d93f32c2263dcbfbfd4\": container with ID starting with 594189661b69288973f3991e85d91ade2c31f61802de3d93f32c2263dcbfbfd4 not found: ID does not exist"
Sep 06 10:43:46.384212 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.384141    2557 scope.go:117] "RemoveContainer" containerID="4a2db8efc82c9d99547b7fe7afe77cae9109cedc73f61fc4d9a3b9790bda25e5"
Sep 06 10:43:46.384542 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:43:46.384521    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a2db8efc82c9d99547b7fe7afe77cae9109cedc73f61fc4d9a3b9790bda25e5\": container with ID starting with 4a2db8efc82c9d99547b7fe7afe77cae9109cedc73f61fc4d9a3b9790bda25e5 not found: ID does not exist" containerID="4a2db8efc82c9d99547b7fe7afe77cae9109cedc73f61fc4d9a3b9790bda25e5"
Sep 06 10:43:46.384612 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.384549    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a2db8efc82c9d99547b7fe7afe77cae9109cedc73f61fc4d9a3b9790bda25e5"} err="failed to get container status \"4a2db8efc82c9d99547b7fe7afe77cae9109cedc73f61fc4d9a3b9790bda25e5\": rpc error: code = NotFound desc = could not find container \"4a2db8efc82c9d99547b7fe7afe77cae9109cedc73f61fc4d9a3b9790bda25e5\": container with ID starting with 4a2db8efc82c9d99547b7fe7afe77cae9109cedc73f61fc4d9a3b9790bda25e5 not found: ID does not exist"
Sep 06 10:43:46.386603 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.386581    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"]
Sep 06 10:43:46.389119 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:46.389095    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-02ec029f-openshift-default-545d7b5b56-p482f"]
Sep 06 10:43:47.709947 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:47.709912    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:43:48.037871 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:48.037788    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84520080-e92f-4385-8165-9689c7a7ebd8" path="/var/lib/kubelet/pods/84520080-e92f-4385-8165-9689c7a7ebd8/volumes"
Sep 06 10:43:49.650362 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:49.650328    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"]
Sep 06 10:43:49.650801 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:49.650581    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc" podUID="83ae9b0d-c656-429b-a014-d09481616a67" containerName="istio-proxy" containerID="cri-o://3869f329a65f3734c76fc237b476920feca4cab30ba522baef613d56a15a710e" gracePeriod=30
Sep 06 10:43:50.403942 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.403408    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"]
Sep 06 10:43:50.403942 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.403896    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84520080-e92f-4385-8165-9689c7a7ebd8" containerName="istio-proxy"
Sep 06 10:43:50.403942 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.403911    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="84520080-e92f-4385-8165-9689c7a7ebd8" containerName="istio-proxy"
Sep 06 10:43:50.403942 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.403920    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6551e8f-6562-462a-aebc-c43c7fb98685" containerName="revoke-keys"
Sep 06 10:43:50.403942 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.403928    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6551e8f-6562-462a-aebc-c43c7fb98685" containerName="revoke-keys"
Sep 06 10:43:50.404347 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.404011    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="84520080-e92f-4385-8165-9689c7a7ebd8" containerName="istio-proxy"
Sep 06 10:43:50.404347 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.404022    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6551e8f-6562-462a-aebc-c43c7fb98685" containerName="revoke-keys"
Sep 06 10:43:50.408683 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.408658    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.411096 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.411071    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-ccf833-openshift-default-dockercfg-vgd7b\""
Sep 06 10:43:50.418503 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.418463    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"]
Sep 06 10:43:50.452452 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.452390    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4516b589-28b0-4692-a537-a78fe803f92f-istio-token\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.452452 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.452446    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-istio-envoy\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.452722 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.452484    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4516b589-28b0-4692-a537-a78fe803f92f-istio-podinfo\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.452722 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.452588    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm2rx\" (UniqueName: \"kubernetes.io/projected/4516b589-28b0-4692-a537-a78fe803f92f-kube-api-access-mm2rx\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.452722 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.452656    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-credential-socket\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.452902 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.452742    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-istio-data\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.452902 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.452769    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4516b589-28b0-4692-a537-a78fe803f92f-istiod-ca-cert\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.452902 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.452825    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-workload-socket\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.452902 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.452847    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-workload-certs\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.553378 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.553327    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-workload-socket\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.553378 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.553382    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-workload-certs\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.553697 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.553423    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4516b589-28b0-4692-a537-a78fe803f92f-istio-token\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.553697 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.553446    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-istio-envoy\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.553697 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.553567    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4516b589-28b0-4692-a537-a78fe803f92f-istio-podinfo\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.553697 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.553647    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm2rx\" (UniqueName: \"kubernetes.io/projected/4516b589-28b0-4692-a537-a78fe803f92f-kube-api-access-mm2rx\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.553934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.553713    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-credential-socket\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.553934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.553818    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-workload-socket\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.553934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.553820    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-istio-data\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.553934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.553857    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4516b589-28b0-4692-a537-a78fe803f92f-istiod-ca-cert\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.553934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.553854    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-workload-certs\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.554280 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.554235    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-istio-data\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.554479 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.554456    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-credential-socket\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.554832 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.554784    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4516b589-28b0-4692-a537-a78fe803f92f-istiod-ca-cert\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.556456 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.556421    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-istio-envoy\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.556623 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.556604    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4516b589-28b0-4692-a537-a78fe803f92f-istio-podinfo\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.565576 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.565550    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm2rx\" (UniqueName: \"kubernetes.io/projected/4516b589-28b0-4692-a537-a78fe803f92f-kube-api-access-mm2rx\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.566345 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.566324    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4516b589-28b0-4692-a537-a78fe803f92f-istio-token\") pod \"e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") " pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:50.722539 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:50.722432    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:51.005001 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:51.004966    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"]
Sep 06 10:43:51.006774 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:43:51.006748    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4516b589_28b0_4692_a537_a78fe803f92f.slice/crio-874972e7228a7125ccef3ba10a4d88d0e50bf9064f1070617a161f9333ed936f WatchSource:0}: Error finding container 874972e7228a7125ccef3ba10a4d88d0e50bf9064f1070617a161f9333ed936f: Status 404 returned error can't find the container with id 874972e7228a7125ccef3ba10a4d88d0e50bf9064f1070617a161f9333ed936f
Sep 06 10:43:51.008868 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:51.008836    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:43:51.008926 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:51.008900    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:43:51.008965 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:51.008930    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:43:51.398879 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:51.398847    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx" event={"ID":"4516b589-28b0-4692-a537-a78fe803f92f","Type":"ContainerStarted","Data":"009defe0724ede7737fc8b3b33530325e13db64127a651eedc381a5ed47afa2d"}
Sep 06 10:43:51.399048 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:51.398897    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx" event={"ID":"4516b589-28b0-4692-a537-a78fe803f92f","Type":"ContainerStarted","Data":"874972e7228a7125ccef3ba10a4d88d0e50bf9064f1070617a161f9333ed936f"}
Sep 06 10:43:51.419814 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:51.419767    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx" podStartSLOduration=1.419751836 podStartE2EDuration="1.419751836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:43:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:43:51.417655605 +0000 UTC m=+843.867436384" watchObservedRunningTime="2026-09-06 10:43:51.419751836 +0000 UTC m=+843.869532604"
Sep 06 10:43:51.722820 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:51.722710    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:52.728636 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:52.728608    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:52.750903 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:52.750870    2557 patch_prober.go:28] interesting pod/e2e-autores-55111c-openshift-default-84675d875d-f58sc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 06 10:43:52.751039 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:52.750954    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc" podUID="83ae9b0d-c656-429b-a014-d09481616a67" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 10:43:53.410544 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:53.410512    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:53.411535 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:53.411516    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:43:54.911637 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:54.911610    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:43:55.003096 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.003011    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83ae9b0d-c656-429b-a014-d09481616a67-istio-token\") pod \"83ae9b0d-c656-429b-a014-d09481616a67\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") "
Sep 06 10:43:55.003096 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.003064    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-workload-socket\") pod \"83ae9b0d-c656-429b-a014-d09481616a67\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") "
Sep 06 10:43:55.003325 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.003100    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-credential-socket\") pod \"83ae9b0d-c656-429b-a014-d09481616a67\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") "
Sep 06 10:43:55.003325 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.003150    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-istio-data\") pod \"83ae9b0d-c656-429b-a014-d09481616a67\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") "
Sep 06 10:43:55.003325 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.003181    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83ae9b0d-c656-429b-a014-d09481616a67-istio-podinfo\") pod \"83ae9b0d-c656-429b-a014-d09481616a67\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") "
Sep 06 10:43:55.003325 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.003242    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83ae9b0d-c656-429b-a014-d09481616a67-istiod-ca-cert\") pod \"83ae9b0d-c656-429b-a014-d09481616a67\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") "
Sep 06 10:43:55.003559 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.003398    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "83ae9b0d-c656-429b-a014-d09481616a67" (UID: "83ae9b0d-c656-429b-a014-d09481616a67"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:43:55.003559 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.003441    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-istio-data" (OuterVolumeSpecName: "istio-data") pod "83ae9b0d-c656-429b-a014-d09481616a67" (UID: "83ae9b0d-c656-429b-a014-d09481616a67"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:43:55.003559 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.003453    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "83ae9b0d-c656-429b-a014-d09481616a67" (UID: "83ae9b0d-c656-429b-a014-d09481616a67"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:43:55.003559 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.003280    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxf45\" (UniqueName: \"kubernetes.io/projected/83ae9b0d-c656-429b-a014-d09481616a67-kube-api-access-fxf45\") pod \"83ae9b0d-c656-429b-a014-d09481616a67\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") "
Sep 06 10:43:55.003761 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.003571    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-workload-certs\") pod \"83ae9b0d-c656-429b-a014-d09481616a67\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") "
Sep 06 10:43:55.003761 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.003612    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-istio-envoy\") pod \"83ae9b0d-c656-429b-a014-d09481616a67\" (UID: \"83ae9b0d-c656-429b-a014-d09481616a67\") "
Sep 06 10:43:55.003761 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.003679    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83ae9b0d-c656-429b-a014-d09481616a67-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "83ae9b0d-c656-429b-a014-d09481616a67" (UID: "83ae9b0d-c656-429b-a014-d09481616a67"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:43:55.003974 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.003916    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "83ae9b0d-c656-429b-a014-d09481616a67" (UID: "83ae9b0d-c656-429b-a014-d09481616a67"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:43:55.004041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.003980    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-workload-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:55.004041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.003994    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-credential-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:55.004041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.004003    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-istio-data\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:55.004041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.004013    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83ae9b0d-c656-429b-a014-d09481616a67-istiod-ca-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:55.005707 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.005675    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83ae9b0d-c656-429b-a014-d09481616a67-istio-token" (OuterVolumeSpecName: "istio-token") pod "83ae9b0d-c656-429b-a014-d09481616a67" (UID: "83ae9b0d-c656-429b-a014-d09481616a67"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:43:55.005707 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.005682    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83ae9b0d-c656-429b-a014-d09481616a67-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "83ae9b0d-c656-429b-a014-d09481616a67" (UID: "83ae9b0d-c656-429b-a014-d09481616a67"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 10:43:55.005977 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.005959    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "83ae9b0d-c656-429b-a014-d09481616a67" (UID: "83ae9b0d-c656-429b-a014-d09481616a67"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:43:55.006364 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.006282    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83ae9b0d-c656-429b-a014-d09481616a67-kube-api-access-fxf45" (OuterVolumeSpecName: "kube-api-access-fxf45") pod "83ae9b0d-c656-429b-a014-d09481616a67" (UID: "83ae9b0d-c656-429b-a014-d09481616a67"). InnerVolumeSpecName "kube-api-access-fxf45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:43:55.104882 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.104847    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83ae9b0d-c656-429b-a014-d09481616a67-istio-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:55.104882 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.104878    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83ae9b0d-c656-429b-a014-d09481616a67-istio-podinfo\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:55.104882 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.104887    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxf45\" (UniqueName: \"kubernetes.io/projected/83ae9b0d-c656-429b-a014-d09481616a67-kube-api-access-fxf45\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:55.105127 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.104896    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-workload-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:55.105127 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.104904    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83ae9b0d-c656-429b-a014-d09481616a67-istio-envoy\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:43:55.422658 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.422623    2557 generic.go:356] "Generic (PLEG): container finished" podID="83ae9b0d-c656-429b-a014-d09481616a67" containerID="3869f329a65f3734c76fc237b476920feca4cab30ba522baef613d56a15a710e" exitCode=0
Sep 06 10:43:55.422902 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.422674    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"
Sep 06 10:43:55.422902 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.422711    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc" event={"ID":"83ae9b0d-c656-429b-a014-d09481616a67","Type":"ContainerDied","Data":"3869f329a65f3734c76fc237b476920feca4cab30ba522baef613d56a15a710e"}
Sep 06 10:43:55.422902 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.422776    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc" event={"ID":"83ae9b0d-c656-429b-a014-d09481616a67","Type":"ContainerDied","Data":"5479b8ac1c6b276fca179da80b5f4185e9b2d500bdf1842799172a7b78c638c3"}
Sep 06 10:43:55.422902 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.422807    2557 scope.go:117] "RemoveContainer" containerID="3869f329a65f3734c76fc237b476920feca4cab30ba522baef613d56a15a710e"
Sep 06 10:43:55.432590 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.432572    2557 scope.go:117] "RemoveContainer" containerID="a017d0f330abe2326600a3f00c7cbb43db5e674e91f9980659e264d6af8dd369"
Sep 06 10:43:55.441220 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.441202    2557 scope.go:117] "RemoveContainer" containerID="3869f329a65f3734c76fc237b476920feca4cab30ba522baef613d56a15a710e"
Sep 06 10:43:55.441531 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:43:55.441506    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3869f329a65f3734c76fc237b476920feca4cab30ba522baef613d56a15a710e\": container with ID starting with 3869f329a65f3734c76fc237b476920feca4cab30ba522baef613d56a15a710e not found: ID does not exist" containerID="3869f329a65f3734c76fc237b476920feca4cab30ba522baef613d56a15a710e"
Sep 06 10:43:55.441584 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.441541    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3869f329a65f3734c76fc237b476920feca4cab30ba522baef613d56a15a710e"} err="failed to get container status \"3869f329a65f3734c76fc237b476920feca4cab30ba522baef613d56a15a710e\": rpc error: code = NotFound desc = could not find container \"3869f329a65f3734c76fc237b476920feca4cab30ba522baef613d56a15a710e\": container with ID starting with 3869f329a65f3734c76fc237b476920feca4cab30ba522baef613d56a15a710e not found: ID does not exist"
Sep 06 10:43:55.441584 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.441557    2557 scope.go:117] "RemoveContainer" containerID="a017d0f330abe2326600a3f00c7cbb43db5e674e91f9980659e264d6af8dd369"
Sep 06 10:43:55.441809 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:43:55.441794    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a017d0f330abe2326600a3f00c7cbb43db5e674e91f9980659e264d6af8dd369\": container with ID starting with a017d0f330abe2326600a3f00c7cbb43db5e674e91f9980659e264d6af8dd369 not found: ID does not exist" containerID="a017d0f330abe2326600a3f00c7cbb43db5e674e91f9980659e264d6af8dd369"
Sep 06 10:43:55.441858 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.441815    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a017d0f330abe2326600a3f00c7cbb43db5e674e91f9980659e264d6af8dd369"} err="failed to get container status \"a017d0f330abe2326600a3f00c7cbb43db5e674e91f9980659e264d6af8dd369\": rpc error: code = NotFound desc = could not find container \"a017d0f330abe2326600a3f00c7cbb43db5e674e91f9980659e264d6af8dd369\": container with ID starting with a017d0f330abe2326600a3f00c7cbb43db5e674e91f9980659e264d6af8dd369 not found: ID does not exist"
Sep 06 10:43:55.445720 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.445695    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"]
Sep 06 10:43:55.449680 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:55.449658    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-55111c-openshift-default-84675d875d-f58sc"]
Sep 06 10:43:56.036824 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:56.036791    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83ae9b0d-c656-429b-a014-d09481616a67" path="/var/lib/kubelet/pods/83ae9b0d-c656-429b-a014-d09481616a67/volumes"
Sep 06 10:43:59.327811 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:43:59.327774    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:44:09.411294 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:09.411243    2557 patch_prober.go:28] interesting pod/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 06 10:44:09.411874 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:09.411325    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx" podUID="4516b589-28b0-4692-a537-a78fe803f92f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 06 10:44:10.518198 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:10.518162    2557 generic.go:356] "Generic (PLEG): container finished" podID="4516b589-28b0-4692-a537-a78fe803f92f" containerID="009defe0724ede7737fc8b3b33530325e13db64127a651eedc381a5ed47afa2d" exitCode=0
Sep 06 10:44:10.518661 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:10.518239    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx" event={"ID":"4516b589-28b0-4692-a537-a78fe803f92f","Type":"ContainerDied","Data":"009defe0724ede7737fc8b3b33530325e13db64127a651eedc381a5ed47afa2d"}
Sep 06 10:44:10.518749 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:10.518734    2557 scope.go:117] "RemoveContainer" containerID="009defe0724ede7737fc8b3b33530325e13db64127a651eedc381a5ed47afa2d"
Sep 06 10:44:10.519912 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:10.519885    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:44:10.519974 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:10.519952    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:44:10.520015 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:10.519983    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:44:11.526111 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:11.526078    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx" event={"ID":"4516b589-28b0-4692-a537-a78fe803f92f","Type":"ContainerStarted","Data":"621e442c1544e18453b2ddc968f8dc7e1e42d9161ebb394a485829044fc36284"}
Sep 06 10:44:11.722832 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:11.722791    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:44:11.723053 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:11.723030    2557 patch_prober.go:28] interesting pod/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 06 10:44:11.723100 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:11.723079    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx" podUID="4516b589-28b0-4692-a537-a78fe803f92f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 06 10:44:12.723305 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:12.723268    2557 patch_prober.go:28] interesting pod/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 06 10:44:12.723707 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:12.723338    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx" podUID="4516b589-28b0-4692-a537-a78fe803f92f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 06 10:44:13.722891 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:13.722854    2557 patch_prober.go:28] interesting pod/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 06 10:44:13.723076 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:13.722915    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx" podUID="4516b589-28b0-4692-a537-a78fe803f92f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 06 10:44:14.723281 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:14.723238    2557 patch_prober.go:28] interesting pod/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 06 10:44:14.723713 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:14.723301    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx" podUID="4516b589-28b0-4692-a537-a78fe803f92f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 06 10:44:15.723100 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:15.723065    2557 patch_prober.go:28] interesting pod/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 06 10:44:15.723347 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:15.723318    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx" podUID="4516b589-28b0-4692-a537-a78fe803f92f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 06 10:44:16.727312 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:16.727282    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:44:16.727709 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:16.727522    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:44:16.728364 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:16.728347    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:44:18.363642 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:18.363607    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:44:30.077366 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:30.077330    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-ccf833-bz8cx"]
Sep 06 10:44:30.077867 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:30.077713    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83ae9b0d-c656-429b-a014-d09481616a67" containerName="istio-proxy"
Sep 06 10:44:30.077867 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:30.077725    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="83ae9b0d-c656-429b-a014-d09481616a67" containerName="istio-proxy"
Sep 06 10:44:30.077867 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:30.077760    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84520080-e92f-4385-8165-9689c7a7ebd8" containerName="istio-proxy"
Sep 06 10:44:30.077867 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:30.077765    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="84520080-e92f-4385-8165-9689c7a7ebd8" containerName="istio-proxy"
Sep 06 10:44:30.077867 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:30.077776    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83ae9b0d-c656-429b-a014-d09481616a67" containerName="istio-proxy"
Sep 06 10:44:30.077867 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:30.077782    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="83ae9b0d-c656-429b-a014-d09481616a67" containerName="istio-proxy"
Sep 06 10:44:30.077867 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:30.077838    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="83ae9b0d-c656-429b-a014-d09481616a67" containerName="istio-proxy"
Sep 06 10:44:30.077867 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:30.077850    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="84520080-e92f-4385-8165-9689c7a7ebd8" containerName="istio-proxy"
Sep 06 10:44:30.081223 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:30.081206    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-ccf833-bz8cx"
Sep 06 10:44:30.083522 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:30.083471    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\""
Sep 06 10:44:30.083664 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:30.083523    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 10:44:30.086587 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:30.086562    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-ccf833-bz8cx"]
Sep 06 10:44:30.132275 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:30.132252    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71010e99-3191-4f23-8c25-ad2293da8b57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-ccf833-bz8cx\" (UID: \"71010e99-3191-4f23-8c25-ad2293da8b57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-ccf833-bz8cx"
Sep 06 10:44:30.233214 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:30.233184    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71010e99-3191-4f23-8c25-ad2293da8b57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-ccf833-bz8cx\" (UID: \"71010e99-3191-4f23-8c25-ad2293da8b57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-ccf833-bz8cx"
Sep 06 10:44:30.233836 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:30.233816    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71010e99-3191-4f23-8c25-ad2293da8b57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-ccf833-bz8cx\" (UID: \"71010e99-3191-4f23-8c25-ad2293da8b57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-ccf833-bz8cx"
Sep 06 10:44:30.392556 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:30.392531    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-ccf833-bz8cx"
Sep 06 10:44:30.669155 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:30.668995    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-ccf833-bz8cx"]
Sep 06 10:44:30.672011 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:44:30.671982    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71010e99_3191_4f23_8c25_ad2293da8b57.slice/crio-c981b65b747fd48d81c7c4add5b7fd71e4bc67d87bd3469951692ebeed864f68 WatchSource:0}: Error finding container c981b65b747fd48d81c7c4add5b7fd71e4bc67d87bd3469951692ebeed864f68: Status 404 returned error can't find the container with id c981b65b747fd48d81c7c4add5b7fd71e4bc67d87bd3469951692ebeed864f68
Sep 06 10:44:31.638730 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:31.638636    2557 generic.go:356] "Generic (PLEG): container finished" podID="71010e99-3191-4f23-8c25-ad2293da8b57" containerID="bc0dcf7195f69916ab5cf99eb3bd8e3d2e03b7a5180c6f9aab72116f6f8dd1a4" exitCode=0
Sep 06 10:44:31.638730 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:31.638680    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-ccf833-bz8cx" event={"ID":"71010e99-3191-4f23-8c25-ad2293da8b57","Type":"ContainerDied","Data":"bc0dcf7195f69916ab5cf99eb3bd8e3d2e03b7a5180c6f9aab72116f6f8dd1a4"}
Sep 06 10:44:31.639107 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:31.638747    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-ccf833-bz8cx" event={"ID":"71010e99-3191-4f23-8c25-ad2293da8b57","Type":"ContainerStarted","Data":"c981b65b747fd48d81c7c4add5b7fd71e4bc67d87bd3469951692ebeed864f68"}
Sep 06 10:44:32.779607 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:32.779583    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-ccf833-bz8cx"
Sep 06 10:44:32.855009 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:32.854980    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71010e99-3191-4f23-8c25-ad2293da8b57-maas-api-service-ca\") pod \"71010e99-3191-4f23-8c25-ad2293da8b57\" (UID: \"71010e99-3191-4f23-8c25-ad2293da8b57\") "
Sep 06 10:44:32.855334 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:32.855309    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71010e99-3191-4f23-8c25-ad2293da8b57-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "71010e99-3191-4f23-8c25-ad2293da8b57" (UID: "71010e99-3191-4f23-8c25-ad2293da8b57"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:44:32.956388 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:32.956323    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71010e99-3191-4f23-8c25-ad2293da8b57-maas-api-service-ca\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:44:33.012195 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:33.012163    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:44:33.650528 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:33.650485    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-ccf833-bz8cx"
Sep 06 10:44:33.650528 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:33.650523    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-ccf833-bz8cx" event={"ID":"71010e99-3191-4f23-8c25-ad2293da8b57","Type":"ContainerDied","Data":"c981b65b747fd48d81c7c4add5b7fd71e4bc67d87bd3469951692ebeed864f68"}
Sep 06 10:44:33.650788 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:33.650571    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c981b65b747fd48d81c7c4add5b7fd71e4bc67d87bd3469951692ebeed864f68"
Sep 06 10:44:40.723241 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:40.723165    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-ccf833-bz8cx"]
Sep 06 10:44:40.726177 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:40.726153    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-ccf833-bz8cx"]
Sep 06 10:44:42.038039 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:42.037996    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71010e99-3191-4f23-8c25-ad2293da8b57" path="/var/lib/kubelet/pods/71010e99-3191-4f23-8c25-ad2293da8b57/volumes"
Sep 06 10:44:47.451199 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:47.451168    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"]
Sep 06 10:44:47.451641 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:47.451383    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx" podUID="4516b589-28b0-4692-a537-a78fe803f92f" containerName="istio-proxy" containerID="cri-o://621e442c1544e18453b2ddc968f8dc7e1e42d9161ebb394a485829044fc36284" gracePeriod=30
Sep 06 10:44:47.825064 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:47.824977    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:44:48.047252 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.047225    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/2.log"
Sep 06 10:44:48.050011 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.049989    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/2.log"
Sep 06 10:44:48.178524 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.178472    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"]
Sep 06 10:44:48.179041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.179017    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71010e99-3191-4f23-8c25-ad2293da8b57" containerName="revoke-keys"
Sep 06 10:44:48.179041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.179040    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="71010e99-3191-4f23-8c25-ad2293da8b57" containerName="revoke-keys"
Sep 06 10:44:48.179235 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.179178    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="83ae9b0d-c656-429b-a014-d09481616a67" containerName="istio-proxy"
Sep 06 10:44:48.179235 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.179192    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="71010e99-3191-4f23-8c25-ad2293da8b57" containerName="revoke-keys"
Sep 06 10:44:48.183627 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.183609    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.186021 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.186003    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-77b0cb-openshift-default-dockercfg-9c8dh\""
Sep 06 10:44:48.193284 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.193258    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"]
Sep 06 10:44:48.295124 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.295090    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-envoy\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.295124 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.295129    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-token\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.295397 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.295162    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwqzl\" (UniqueName: \"kubernetes.io/projected/b804996f-c5f4-46b7-a501-39a9453e12fb-kube-api-access-hwqzl\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.295397 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.295240    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-workload-socket\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.295397 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.295291    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-data\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.295397 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.295313    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-workload-certs\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.295643 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.295404    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-podinfo\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.295643 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.295446    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-credential-socket\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.295643 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.295615    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b804996f-c5f4-46b7-a501-39a9453e12fb-istiod-ca-cert\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.396801 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.396692    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-data\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.396801 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.396740    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-workload-certs\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.397047 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.396851    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-podinfo\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.397047 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.396897    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-credential-socket\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.397047 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.396972    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b804996f-c5f4-46b7-a501-39a9453e12fb-istiod-ca-cert\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.397047 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.397009    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-envoy\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.397047 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.397026    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-token\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.397047 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.397048    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwqzl\" (UniqueName: \"kubernetes.io/projected/b804996f-c5f4-46b7-a501-39a9453e12fb-kube-api-access-hwqzl\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.397338 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.397097    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-workload-socket\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.397338 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.397121    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-data\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.397338 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.397176    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-workload-certs\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.397338 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.397265    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-credential-socket\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.397572 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.397423    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-workload-socket\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.397935 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.397909    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b804996f-c5f4-46b7-a501-39a9453e12fb-istiod-ca-cert\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.399155 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.399135    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-envoy\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.399421 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.399405    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-podinfo\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.404619 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.404594    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-token\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.404839 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.404810    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwqzl\" (UniqueName: \"kubernetes.io/projected/b804996f-c5f4-46b7-a501-39a9453e12fb-kube-api-access-hwqzl\") pod \"e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") " pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.494988 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.494947    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:48.781635 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.781608    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"]
Sep 06 10:44:48.784226 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:44:48.784183    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb804996f_c5f4_46b7_a501_39a9453e12fb.slice/crio-5841660e02e2b3030abc66d09c4007d69099a76c5f022e4a5b167c30d296ab63 WatchSource:0}: Error finding container 5841660e02e2b3030abc66d09c4007d69099a76c5f022e4a5b167c30d296ab63: Status 404 returned error can't find the container with id 5841660e02e2b3030abc66d09c4007d69099a76c5f022e4a5b167c30d296ab63
Sep 06 10:44:48.786823 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.786779    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:44:48.786905 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.786871    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:44:48.786955 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:48.786917    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:44:49.737162 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:49.737125    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4" event={"ID":"b804996f-c5f4-46b7-a501-39a9453e12fb","Type":"ContainerStarted","Data":"0ea9adf11003a27641ee5d1685ff56e37389a51bdc73c7b5d1822e6a8e49465b"}
Sep 06 10:44:49.737562 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:49.737177    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4" event={"ID":"b804996f-c5f4-46b7-a501-39a9453e12fb","Type":"ContainerStarted","Data":"5841660e02e2b3030abc66d09c4007d69099a76c5f022e4a5b167c30d296ab63"}
Sep 06 10:44:49.755975 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:49.755914    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4" podStartSLOduration=1.755897648 podStartE2EDuration="1.755897648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:44:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:44:49.754482308 +0000 UTC m=+902.204263075" watchObservedRunningTime="2026-09-06 10:44:49.755897648 +0000 UTC m=+902.205678415"
Sep 06 10:44:50.495564 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:50.495516    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:50.500932 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:50.500900    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:50.742655 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:50.742622    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:50.743607 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:50.743587    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:44:52.696067 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.696041    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:44:52.755400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.755317    2557 generic.go:356] "Generic (PLEG): container finished" podID="4516b589-28b0-4692-a537-a78fe803f92f" containerID="621e442c1544e18453b2ddc968f8dc7e1e42d9161ebb394a485829044fc36284" exitCode=0
Sep 06 10:44:52.755400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.755367    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"
Sep 06 10:44:52.755622 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.755402    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx" event={"ID":"4516b589-28b0-4692-a537-a78fe803f92f","Type":"ContainerDied","Data":"621e442c1544e18453b2ddc968f8dc7e1e42d9161ebb394a485829044fc36284"}
Sep 06 10:44:52.755622 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.755452    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx" event={"ID":"4516b589-28b0-4692-a537-a78fe803f92f","Type":"ContainerDied","Data":"874972e7228a7125ccef3ba10a4d88d0e50bf9064f1070617a161f9333ed936f"}
Sep 06 10:44:52.755622 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.755518    2557 scope.go:117] "RemoveContainer" containerID="621e442c1544e18453b2ddc968f8dc7e1e42d9161ebb394a485829044fc36284"
Sep 06 10:44:52.764332 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.764315    2557 scope.go:117] "RemoveContainer" containerID="009defe0724ede7737fc8b3b33530325e13db64127a651eedc381a5ed47afa2d"
Sep 06 10:44:52.772771 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.772740    2557 scope.go:117] "RemoveContainer" containerID="621e442c1544e18453b2ddc968f8dc7e1e42d9161ebb394a485829044fc36284"
Sep 06 10:44:52.773058 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:44:52.773039    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"621e442c1544e18453b2ddc968f8dc7e1e42d9161ebb394a485829044fc36284\": container with ID starting with 621e442c1544e18453b2ddc968f8dc7e1e42d9161ebb394a485829044fc36284 not found: ID does not exist" containerID="621e442c1544e18453b2ddc968f8dc7e1e42d9161ebb394a485829044fc36284"
Sep 06 10:44:52.773120 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.773067    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"621e442c1544e18453b2ddc968f8dc7e1e42d9161ebb394a485829044fc36284"} err="failed to get container status \"621e442c1544e18453b2ddc968f8dc7e1e42d9161ebb394a485829044fc36284\": rpc error: code = NotFound desc = could not find container \"621e442c1544e18453b2ddc968f8dc7e1e42d9161ebb394a485829044fc36284\": container with ID starting with 621e442c1544e18453b2ddc968f8dc7e1e42d9161ebb394a485829044fc36284 not found: ID does not exist"
Sep 06 10:44:52.773120 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.773084    2557 scope.go:117] "RemoveContainer" containerID="009defe0724ede7737fc8b3b33530325e13db64127a651eedc381a5ed47afa2d"
Sep 06 10:44:52.773338 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:44:52.773320    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"009defe0724ede7737fc8b3b33530325e13db64127a651eedc381a5ed47afa2d\": container with ID starting with 009defe0724ede7737fc8b3b33530325e13db64127a651eedc381a5ed47afa2d not found: ID does not exist" containerID="009defe0724ede7737fc8b3b33530325e13db64127a651eedc381a5ed47afa2d"
Sep 06 10:44:52.773383 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.773343    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"009defe0724ede7737fc8b3b33530325e13db64127a651eedc381a5ed47afa2d"} err="failed to get container status \"009defe0724ede7737fc8b3b33530325e13db64127a651eedc381a5ed47afa2d\": rpc error: code = NotFound desc = could not find container \"009defe0724ede7737fc8b3b33530325e13db64127a651eedc381a5ed47afa2d\": container with ID starting with 009defe0724ede7737fc8b3b33530325e13db64127a651eedc381a5ed47afa2d not found: ID does not exist"
Sep 06 10:44:52.842943 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.842906    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4516b589-28b0-4692-a537-a78fe803f92f-istio-token\") pod \"4516b589-28b0-4692-a537-a78fe803f92f\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") "
Sep 06 10:44:52.843128 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.842957    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-istio-envoy\") pod \"4516b589-28b0-4692-a537-a78fe803f92f\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") "
Sep 06 10:44:52.843128 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.843011    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mm2rx\" (UniqueName: \"kubernetes.io/projected/4516b589-28b0-4692-a537-a78fe803f92f-kube-api-access-mm2rx\") pod \"4516b589-28b0-4692-a537-a78fe803f92f\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") "
Sep 06 10:44:52.843128 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.843042    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-credential-socket\") pod \"4516b589-28b0-4692-a537-a78fe803f92f\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") "
Sep 06 10:44:52.843128 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.843071    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-istio-data\") pod \"4516b589-28b0-4692-a537-a78fe803f92f\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") "
Sep 06 10:44:52.843128 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.843092    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-workload-socket\") pod \"4516b589-28b0-4692-a537-a78fe803f92f\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") "
Sep 06 10:44:52.843384 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.843150    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4516b589-28b0-4692-a537-a78fe803f92f-istio-podinfo\") pod \"4516b589-28b0-4692-a537-a78fe803f92f\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") "
Sep 06 10:44:52.843384 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.843251    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-workload-certs\") pod \"4516b589-28b0-4692-a537-a78fe803f92f\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") "
Sep 06 10:44:52.843384 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.843280    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4516b589-28b0-4692-a537-a78fe803f92f-istiod-ca-cert\") pod \"4516b589-28b0-4692-a537-a78fe803f92f\" (UID: \"4516b589-28b0-4692-a537-a78fe803f92f\") "
Sep 06 10:44:52.843573 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.843411    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4516b589-28b0-4692-a537-a78fe803f92f" (UID: "4516b589-28b0-4692-a537-a78fe803f92f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:44:52.843573 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.843453    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4516b589-28b0-4692-a537-a78fe803f92f" (UID: "4516b589-28b0-4692-a537-a78fe803f92f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:44:52.843573 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.843424    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-istio-data" (OuterVolumeSpecName: "istio-data") pod "4516b589-28b0-4692-a537-a78fe803f92f" (UID: "4516b589-28b0-4692-a537-a78fe803f92f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:44:52.843779 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.843657    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4516b589-28b0-4692-a537-a78fe803f92f" (UID: "4516b589-28b0-4692-a537-a78fe803f92f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:44:52.843843 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.843809    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-workload-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:44:52.843843 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.843831    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-credential-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:44:52.843947 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.843846    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-istio-data\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:44:52.843947 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.843860    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-workload-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:44:52.843947 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.843839    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4516b589-28b0-4692-a537-a78fe803f92f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4516b589-28b0-4692-a537-a78fe803f92f" (UID: "4516b589-28b0-4692-a537-a78fe803f92f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:44:52.845717 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.845688    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4516b589-28b0-4692-a537-a78fe803f92f-kube-api-access-mm2rx" (OuterVolumeSpecName: "kube-api-access-mm2rx") pod "4516b589-28b0-4692-a537-a78fe803f92f" (UID: "4516b589-28b0-4692-a537-a78fe803f92f"). InnerVolumeSpecName "kube-api-access-mm2rx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:44:52.845841 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.845737    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4516b589-28b0-4692-a537-a78fe803f92f-istio-token" (OuterVolumeSpecName: "istio-token") pod "4516b589-28b0-4692-a537-a78fe803f92f" (UID: "4516b589-28b0-4692-a537-a78fe803f92f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:44:52.845841 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.845765    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4516b589-28b0-4692-a537-a78fe803f92f" (UID: "4516b589-28b0-4692-a537-a78fe803f92f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:44:52.845841 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.845826    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4516b589-28b0-4692-a537-a78fe803f92f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4516b589-28b0-4692-a537-a78fe803f92f" (UID: "4516b589-28b0-4692-a537-a78fe803f92f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 10:44:52.944749 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.944712    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4516b589-28b0-4692-a537-a78fe803f92f-istiod-ca-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:44:52.944749 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.944742    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4516b589-28b0-4692-a537-a78fe803f92f-istio-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:44:52.944749 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.944751    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4516b589-28b0-4692-a537-a78fe803f92f-istio-envoy\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:44:52.945015 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.944769    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mm2rx\" (UniqueName: \"kubernetes.io/projected/4516b589-28b0-4692-a537-a78fe803f92f-kube-api-access-mm2rx\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:44:52.945015 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:52.944784    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4516b589-28b0-4692-a537-a78fe803f92f-istio-podinfo\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:44:53.083890 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:53.083828    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"]
Sep 06 10:44:53.084328 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:53.084180    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-ccf833-openshift-default-ff8cb7d7b-88xvx"]
Sep 06 10:44:54.037353 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:44:54.037314    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4516b589-28b0-4692-a537-a78fe803f92f" path="/var/lib/kubelet/pods/4516b589-28b0-4692-a537-a78fe803f92f/volumes"
Sep 06 10:45:00.140299 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.140267    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-232aeb-29811525-b6vcg"]
Sep 06 10:45:00.140698 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.140653    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4516b589-28b0-4692-a537-a78fe803f92f" containerName="istio-proxy"
Sep 06 10:45:00.140698 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.140665    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4516b589-28b0-4692-a537-a78fe803f92f" containerName="istio-proxy"
Sep 06 10:45:00.140698 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.140678    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4516b589-28b0-4692-a537-a78fe803f92f" containerName="istio-proxy"
Sep 06 10:45:00.140698 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.140683    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4516b589-28b0-4692-a537-a78fe803f92f" containerName="istio-proxy"
Sep 06 10:45:00.140830 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.140737    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4516b589-28b0-4692-a537-a78fe803f92f" containerName="istio-proxy"
Sep 06 10:45:00.140830 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.140747    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4516b589-28b0-4692-a537-a78fe803f92f" containerName="istio-proxy"
Sep 06 10:45:00.145181 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.145160    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-636b04-29811525-jmhfd"]
Sep 06 10:45:00.145308 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.145269    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-232aeb-29811525-b6vcg"
Sep 06 10:45:00.147630 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.147611    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\""
Sep 06 10:45:00.148378 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.148364    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-636b04-29811525-jmhfd"
Sep 06 10:45:00.153449 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.153145    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-232aeb-29811525-b6vcg"]
Sep 06 10:45:00.158972 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.158936    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-232aeb-29811525-b6vcg"
Sep 06 10:45:00.168775 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.168747    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-636b04-29811525-jmhfd"
Sep 06 10:45:00.168895 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.168873    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-636b04-29811525-jmhfd"]
Sep 06 10:45:00.231576 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.231543    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:45:00.437562 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.437537    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-232aeb-29811525-b6vcg"]
Sep 06 10:45:00.439588 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:45:00.439560    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44ebfb8c_91a6_48df_842d_493a5df2bdd0.slice/crio-2b21306070294936f3adbe2e1c603799a0c7cb6f4062458dd74e0b45427a0665 WatchSource:0}: Error finding container 2b21306070294936f3adbe2e1c603799a0c7cb6f4062458dd74e0b45427a0665: Status 404 returned error can't find the container with id 2b21306070294936f3adbe2e1c603799a0c7cb6f4062458dd74e0b45427a0665
Sep 06 10:45:00.445249 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.445228    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-636b04-29811525-jmhfd"]
Sep 06 10:45:00.447504 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:45:00.447468    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7b102fb_7829_4eff_a3e5_55facbb633ca.slice/crio-55b10996c9eed008461ea329d282c107590bf17647e35608d7c4320a675cbb58 WatchSource:0}: Error finding container 55b10996c9eed008461ea329d282c107590bf17647e35608d7c4320a675cbb58: Status 404 returned error can't find the container with id 55b10996c9eed008461ea329d282c107590bf17647e35608d7c4320a675cbb58
Sep 06 10:45:00.803719 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.803687    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-232aeb-29811525-b6vcg" event={"ID":"44ebfb8c-91a6-48df-842d-493a5df2bdd0","Type":"ContainerStarted","Data":"1ba27cb1b73d3e648efbe541bae733bbe304168238c71b3bdb37a2b5fb7bf4f7"}
Sep 06 10:45:00.803877 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.803748    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-232aeb-29811525-b6vcg" event={"ID":"44ebfb8c-91a6-48df-842d-493a5df2bdd0","Type":"ContainerStarted","Data":"2b21306070294936f3adbe2e1c603799a0c7cb6f4062458dd74e0b45427a0665"}
Sep 06 10:45:00.805240 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.805214    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-636b04-29811525-jmhfd" event={"ID":"b7b102fb-7829-4eff-a3e5-55facbb633ca","Type":"ContainerStarted","Data":"6494440d9a6013c174965291894b801e882a23f39f2e9836ec37b8d115b1216b"}
Sep 06 10:45:00.805328 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.805269    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-636b04-29811525-jmhfd" event={"ID":"b7b102fb-7829-4eff-a3e5-55facbb633ca","Type":"ContainerStarted","Data":"55b10996c9eed008461ea329d282c107590bf17647e35608d7c4320a675cbb58"}
Sep 06 10:45:00.823095 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.823054    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-232aeb-29811525-b6vcg" podStartSLOduration=0.823039175 podStartE2EDuration="823.039175ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:45:00.821472077 +0000 UTC m=+913.271252844" watchObservedRunningTime="2026-09-06 10:45:00.823039175 +0000 UTC m=+913.272819943"
Sep 06 10:45:00.835544 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:00.835473    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-636b04-29811525-jmhfd" podStartSLOduration=0.835461697 podStartE2EDuration="835.461697ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:45:00.835261626 +0000 UTC m=+913.285042400" watchObservedRunningTime="2026-09-06 10:45:00.835461697 +0000 UTC m=+913.285242466"
Sep 06 10:45:01.699533 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:01.699432    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"]
Sep 06 10:45:01.699920 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:01.699650    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4" podUID="b804996f-c5f4-46b7-a501-39a9453e12fb" containerName="istio-proxy" containerID="cri-o://0ea9adf11003a27641ee5d1685ff56e37389a51bdc73c7b5d1822e6a8e49465b" gracePeriod=30
Sep 06 10:45:01.813583 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:01.813537    2557 generic.go:356] "Generic (PLEG): container finished" podID="44ebfb8c-91a6-48df-842d-493a5df2bdd0" containerID="1ba27cb1b73d3e648efbe541bae733bbe304168238c71b3bdb37a2b5fb7bf4f7" exitCode=0
Sep 06 10:45:01.813887 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:01.813615    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-232aeb-29811525-b6vcg" event={"ID":"44ebfb8c-91a6-48df-842d-493a5df2bdd0","Type":"ContainerDied","Data":"1ba27cb1b73d3e648efbe541bae733bbe304168238c71b3bdb37a2b5fb7bf4f7"}
Sep 06 10:45:01.815723 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:01.815699    2557 generic.go:356] "Generic (PLEG): container finished" podID="b7b102fb-7829-4eff-a3e5-55facbb633ca" containerID="6494440d9a6013c174965291894b801e882a23f39f2e9836ec37b8d115b1216b" exitCode=0
Sep 06 10:45:01.815855 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:01.815749    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-636b04-29811525-jmhfd" event={"ID":"b7b102fb-7829-4eff-a3e5-55facbb633ca","Type":"ContainerDied","Data":"6494440d9a6013c174965291894b801e882a23f39f2e9836ec37b8d115b1216b"}
Sep 06 10:45:02.954048 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:02.954022    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-636b04-29811525-jmhfd"
Sep 06 10:45:02.994400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:02.994377    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-232aeb-29811525-b6vcg"
Sep 06 10:45:03.827540 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:03.827484    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-232aeb-29811525-b6vcg"
Sep 06 10:45:03.827729 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:03.827533    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-232aeb-29811525-b6vcg" event={"ID":"44ebfb8c-91a6-48df-842d-493a5df2bdd0","Type":"ContainerDied","Data":"2b21306070294936f3adbe2e1c603799a0c7cb6f4062458dd74e0b45427a0665"}
Sep 06 10:45:03.827729 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:03.827585    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b21306070294936f3adbe2e1c603799a0c7cb6f4062458dd74e0b45427a0665"
Sep 06 10:45:03.828847 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:03.828828    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-636b04-29811525-jmhfd"
Sep 06 10:45:03.828847 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:03.828834    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-636b04-29811525-jmhfd" event={"ID":"b7b102fb-7829-4eff-a3e5-55facbb633ca","Type":"ContainerDied","Data":"55b10996c9eed008461ea329d282c107590bf17647e35608d7c4320a675cbb58"}
Sep 06 10:45:03.829004 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:03.828869    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55b10996c9eed008461ea329d282c107590bf17647e35608d7c4320a675cbb58"
Sep 06 10:45:05.745212 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:05.745177    2557 patch_prober.go:28] interesting pod/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 06 10:45:05.745622 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:05.745235    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4" podUID="b804996f-c5f4-46b7-a501-39a9453e12fb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 10:45:06.849655 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:06.849623    2557 generic.go:356] "Generic (PLEG): container finished" podID="b804996f-c5f4-46b7-a501-39a9453e12fb" containerID="0ea9adf11003a27641ee5d1685ff56e37389a51bdc73c7b5d1822e6a8e49465b" exitCode=0
Sep 06 10:45:06.850052 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:06.849701    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4" event={"ID":"b804996f-c5f4-46b7-a501-39a9453e12fb","Type":"ContainerDied","Data":"0ea9adf11003a27641ee5d1685ff56e37389a51bdc73c7b5d1822e6a8e49465b"}
Sep 06 10:45:06.954876 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:06.954852    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:45:07.077906 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.077816    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-token\") pod \"b804996f-c5f4-46b7-a501-39a9453e12fb\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") "
Sep 06 10:45:07.078058 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.077924    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-data\") pod \"b804996f-c5f4-46b7-a501-39a9453e12fb\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") "
Sep 06 10:45:07.078058 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.077964    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwqzl\" (UniqueName: \"kubernetes.io/projected/b804996f-c5f4-46b7-a501-39a9453e12fb-kube-api-access-hwqzl\") pod \"b804996f-c5f4-46b7-a501-39a9453e12fb\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") "
Sep 06 10:45:07.078058 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.077995    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-workload-certs\") pod \"b804996f-c5f4-46b7-a501-39a9453e12fb\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") "
Sep 06 10:45:07.078058 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.078013    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-credential-socket\") pod \"b804996f-c5f4-46b7-a501-39a9453e12fb\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") "
Sep 06 10:45:07.078238 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.078061    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-podinfo\") pod \"b804996f-c5f4-46b7-a501-39a9453e12fb\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") "
Sep 06 10:45:07.078238 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.078087    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b804996f-c5f4-46b7-a501-39a9453e12fb-istiod-ca-cert\") pod \"b804996f-c5f4-46b7-a501-39a9453e12fb\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") "
Sep 06 10:45:07.078238 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.078127    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-workload-socket\") pod \"b804996f-c5f4-46b7-a501-39a9453e12fb\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") "
Sep 06 10:45:07.078238 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.078173    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-envoy\") pod \"b804996f-c5f4-46b7-a501-39a9453e12fb\" (UID: \"b804996f-c5f4-46b7-a501-39a9453e12fb\") "
Sep 06 10:45:07.078238 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.078199    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-data" (OuterVolumeSpecName: "istio-data") pod "b804996f-c5f4-46b7-a501-39a9453e12fb" (UID: "b804996f-c5f4-46b7-a501-39a9453e12fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:45:07.078535 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.078282    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b804996f-c5f4-46b7-a501-39a9453e12fb" (UID: "b804996f-c5f4-46b7-a501-39a9453e12fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:45:07.078535 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.078408    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b804996f-c5f4-46b7-a501-39a9453e12fb" (UID: "b804996f-c5f4-46b7-a501-39a9453e12fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:45:07.078708 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.078463    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b804996f-c5f4-46b7-a501-39a9453e12fb" (UID: "b804996f-c5f4-46b7-a501-39a9453e12fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:45:07.078708 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.078592    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b804996f-c5f4-46b7-a501-39a9453e12fb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b804996f-c5f4-46b7-a501-39a9453e12fb" (UID: "b804996f-c5f4-46b7-a501-39a9453e12fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:45:07.078837 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.078568    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-data\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:07.078837 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.078735    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-workload-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:07.080089 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.080067    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-token" (OuterVolumeSpecName: "istio-token") pod "b804996f-c5f4-46b7-a501-39a9453e12fb" (UID: "b804996f-c5f4-46b7-a501-39a9453e12fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:45:07.080178 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.080158    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b804996f-c5f4-46b7-a501-39a9453e12fb-kube-api-access-hwqzl" (OuterVolumeSpecName: "kube-api-access-hwqzl") pod "b804996f-c5f4-46b7-a501-39a9453e12fb" (UID: "b804996f-c5f4-46b7-a501-39a9453e12fb"). InnerVolumeSpecName "kube-api-access-hwqzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:45:07.080730 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.080706    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b804996f-c5f4-46b7-a501-39a9453e12fb" (UID: "b804996f-c5f4-46b7-a501-39a9453e12fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 10:45:07.080804 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.080727    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b804996f-c5f4-46b7-a501-39a9453e12fb" (UID: "b804996f-c5f4-46b7-a501-39a9453e12fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:45:07.179571 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.179535    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-envoy\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:07.179571 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.179563    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:07.179571 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.179572    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwqzl\" (UniqueName: \"kubernetes.io/projected/b804996f-c5f4-46b7-a501-39a9453e12fb-kube-api-access-hwqzl\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:07.179811 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.179583    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-credential-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:07.179811 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.179594    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b804996f-c5f4-46b7-a501-39a9453e12fb-istio-podinfo\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:07.179811 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.179606    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b804996f-c5f4-46b7-a501-39a9453e12fb-istiod-ca-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:07.179811 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.179615    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b804996f-c5f4-46b7-a501-39a9453e12fb-workload-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:07.857041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.857011    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"
Sep 06 10:45:07.857041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.857034    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4" event={"ID":"b804996f-c5f4-46b7-a501-39a9453e12fb","Type":"ContainerDied","Data":"5841660e02e2b3030abc66d09c4007d69099a76c5f022e4a5b167c30d296ab63"}
Sep 06 10:45:07.857580 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.857113    2557 scope.go:117] "RemoveContainer" containerID="0ea9adf11003a27641ee5d1685ff56e37389a51bdc73c7b5d1822e6a8e49465b"
Sep 06 10:45:07.882861 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.882834    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"]
Sep 06 10:45:07.885946 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:07.885921    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-77b0cb-openshift-default-77b4bb4758-6qlv4"]
Sep 06 10:45:08.036934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:08.036897    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b804996f-c5f4-46b7-a501-39a9453e12fb" path="/var/lib/kubelet/pods/b804996f-c5f4-46b7-a501-39a9453e12fb/volumes"
Sep 06 10:45:14.427302 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:14.427271    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-232aeb-cqwnd"]
Sep 06 10:45:14.427752 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:14.427731    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7b102fb-7829-4eff-a3e5-55facbb633ca" containerName="cleanup"
Sep 06 10:45:14.427752 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:14.427749    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7b102fb-7829-4eff-a3e5-55facbb633ca" containerName="cleanup"
Sep 06 10:45:14.427873 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:14.427772    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b804996f-c5f4-46b7-a501-39a9453e12fb" containerName="istio-proxy"
Sep 06 10:45:14.427873 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:14.427781    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b804996f-c5f4-46b7-a501-39a9453e12fb" containerName="istio-proxy"
Sep 06 10:45:14.427873 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:14.427796    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44ebfb8c-91a6-48df-842d-493a5df2bdd0" containerName="cleanup"
Sep 06 10:45:14.427873 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:14.427804    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="44ebfb8c-91a6-48df-842d-493a5df2bdd0" containerName="cleanup"
Sep 06 10:45:14.428076 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:14.427904    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="44ebfb8c-91a6-48df-842d-493a5df2bdd0" containerName="cleanup"
Sep 06 10:45:14.428076 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:14.427919    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7b102fb-7829-4eff-a3e5-55facbb633ca" containerName="cleanup"
Sep 06 10:45:14.428076 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:14.427932    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b804996f-c5f4-46b7-a501-39a9453e12fb" containerName="istio-proxy"
Sep 06 10:45:14.431008 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:14.430992    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-232aeb-cqwnd"
Sep 06 10:45:14.433355 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:14.433325    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 10:45:14.433454 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:14.433436    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\""
Sep 06 10:45:14.438466 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:14.438120    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-232aeb-cqwnd"]
Sep 06 10:45:14.448441 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:14.448419    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-232aeb-cqwnd\" (UID: \"e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-232aeb-cqwnd"
Sep 06 10:45:14.549768 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:14.549733    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-232aeb-cqwnd\" (UID: \"e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-232aeb-cqwnd"
Sep 06 10:45:14.550353 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:14.550333    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-232aeb-cqwnd\" (UID: \"e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-232aeb-cqwnd"
Sep 06 10:45:14.742558 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:14.742466    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-232aeb-cqwnd"
Sep 06 10:45:15.013576 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:15.013546    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-232aeb-cqwnd"]
Sep 06 10:45:15.015928 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:45:15.015904    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5dd36b7_fe6f_4f9a_ad17_62d6a9cd9f86.slice/crio-0ade9890b3e5c5aa11fbfc4012c8c8111f9b8d3c80198f72f7c541a072fb2844 WatchSource:0}: Error finding container 0ade9890b3e5c5aa11fbfc4012c8c8111f9b8d3c80198f72f7c541a072fb2844: Status 404 returned error can't find the container with id 0ade9890b3e5c5aa11fbfc4012c8c8111f9b8d3c80198f72f7c541a072fb2844
Sep 06 10:45:15.903590 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:15.903559    2557 generic.go:356] "Generic (PLEG): container finished" podID="e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86" containerID="e244e2dce5a9142bbaafb05add6fa09755b505773b1f13e782ae2a8270fa4bcb" exitCode=0
Sep 06 10:45:15.903948 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:15.903646    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-232aeb-cqwnd" event={"ID":"e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86","Type":"ContainerDied","Data":"e244e2dce5a9142bbaafb05add6fa09755b505773b1f13e782ae2a8270fa4bcb"}
Sep 06 10:45:15.903948 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:15.903698    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-232aeb-cqwnd" event={"ID":"e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86","Type":"ContainerStarted","Data":"0ade9890b3e5c5aa11fbfc4012c8c8111f9b8d3c80198f72f7c541a072fb2844"}
Sep 06 10:45:16.317973 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.317936    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"]
Sep 06 10:45:16.321758 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.321735    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.324500 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.324472    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2313d276-openshift-default-dockercfg-r4vhx\""
Sep 06 10:45:16.334128 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.334104    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"]
Sep 06 10:45:16.366398 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.366359    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-credential-socket\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.366538 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.366433    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-data\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.366538 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.366476    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-podinfo\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.366618 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.366564    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-workload-socket\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.366618 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.366591    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istiod-ca-cert\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.366681 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.366625    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzlxg\" (UniqueName: \"kubernetes.io/projected/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-kube-api-access-vzlxg\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.366681 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.366657    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-token\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.366773 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.366741    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-envoy\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.366811 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.366785    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-workload-certs\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.467537 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.467500    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-workload-socket\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.467537 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.467543    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istiod-ca-cert\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.467789 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.467566    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzlxg\" (UniqueName: \"kubernetes.io/projected/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-kube-api-access-vzlxg\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.467789 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.467586    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-token\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.467789 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.467632    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-envoy\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.467789 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.467659    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-workload-certs\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.467789 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.467739    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-credential-socket\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.467789 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.467781    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-data\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.468097 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.467819    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-podinfo\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.468097 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.467918    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-workload-socket\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.468213 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.468094    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-workload-certs\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.468213 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.468186    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-data\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.468321 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.468245    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-credential-socket\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.468711 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.468688    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istiod-ca-cert\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.470155 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.470125    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-envoy\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.470357 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.470323    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-podinfo\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.475335 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.475310    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-token\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.475528 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.475510    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzlxg\" (UniqueName: \"kubernetes.io/projected/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-kube-api-access-vzlxg\") pod \"e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") " pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.634819 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.634789    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:16.910091 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.910068    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"]
Sep 06 10:45:16.912214 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:45:16.912185    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf0c73e7_c6e2_47b5_a2b9_daaaed78af36.slice/crio-22f194b2a24572550dddd5590ebda0508a2c9f8b0f2cb7e0939858df39614d46 WatchSource:0}: Error finding container 22f194b2a24572550dddd5590ebda0508a2c9f8b0f2cb7e0939858df39614d46: Status 404 returned error can't find the container with id 22f194b2a24572550dddd5590ebda0508a2c9f8b0f2cb7e0939858df39614d46
Sep 06 10:45:16.914938 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.914903    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:45:16.915043 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.914967    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:45:16.915043 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:16.914997    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:45:17.031704 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:17.031675    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-232aeb-cqwnd"
Sep 06 10:45:17.072707 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:17.072674    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86-maas-api-service-ca\") pod \"e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86\" (UID: \"e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86\") "
Sep 06 10:45:17.073072 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:17.073046    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86" (UID: "e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:45:17.174335 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:17.174243    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86-maas-api-service-ca\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:17.917547 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:17.917474    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k" event={"ID":"df0c73e7-c6e2-47b5-a2b9-daaaed78af36","Type":"ContainerStarted","Data":"1b4f8f1501220fd735928c98a575b617eeebc238a84e8085dffb11f0eea84522"}
Sep 06 10:45:17.917955 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:17.917583    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k" event={"ID":"df0c73e7-c6e2-47b5-a2b9-daaaed78af36","Type":"ContainerStarted","Data":"22f194b2a24572550dddd5590ebda0508a2c9f8b0f2cb7e0939858df39614d46"}
Sep 06 10:45:17.918829 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:17.918805    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-232aeb-cqwnd" event={"ID":"e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86","Type":"ContainerDied","Data":"0ade9890b3e5c5aa11fbfc4012c8c8111f9b8d3c80198f72f7c541a072fb2844"}
Sep 06 10:45:17.918941 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:17.918831    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-232aeb-cqwnd"
Sep 06 10:45:17.918941 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:17.918840    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ade9890b3e5c5aa11fbfc4012c8c8111f9b8d3c80198f72f7c541a072fb2844"
Sep 06 10:45:17.939171 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:17.939117    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k" podStartSLOduration=1.939102782 podStartE2EDuration="1.939102782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:45:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:45:17.937477476 +0000 UTC m=+930.387258256" watchObservedRunningTime="2026-09-06 10:45:17.939102782 +0000 UTC m=+930.388883549"
Sep 06 10:45:18.635888 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:18.635849    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:18.641136 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:18.641110    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:18.910925 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:18.910830    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:45:18.924300 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:18.924275    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:18.925261 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:18.925242    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:25.109287 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:25.109254    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-232aeb-29811525-b6vcg"]
Sep 06 10:45:25.111136 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:25.111115    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-232aeb-29811525-b6vcg"]
Sep 06 10:45:25.315242 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:25.315211    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-232aeb-cqwnd"]
Sep 06 10:45:25.320756 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:25.320729    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-232aeb-cqwnd"]
Sep 06 10:45:26.036863 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:26.036829    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44ebfb8c-91a6-48df-842d-493a5df2bdd0" path="/var/lib/kubelet/pods/44ebfb8c-91a6-48df-842d-493a5df2bdd0/volumes"
Sep 06 10:45:26.037129 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:26.037115    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86" path="/var/lib/kubelet/pods/e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86/volumes"
Sep 06 10:45:30.737598 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:30.737555    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:45:32.115912 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:32.115875    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-636b04-7m58c"]
Sep 06 10:45:32.116287 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:32.116258    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86" containerName="revoke-keys"
Sep 06 10:45:32.116287 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:32.116269    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86" containerName="revoke-keys"
Sep 06 10:45:32.116365 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:32.116333    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5dd36b7-fe6f-4f9a-ad17-62d6a9cd9f86" containerName="revoke-keys"
Sep 06 10:45:32.120692 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:32.120667    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-636b04-7m58c"
Sep 06 10:45:32.123038 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:32.123005    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 10:45:32.124564 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:32.124538    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\""
Sep 06 10:45:32.133505 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:32.129396    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-636b04-7m58c"]
Sep 06 10:45:32.222266 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:32.222230    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9e3bd68-a39f-49ad-8ab2-4baa27988e22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-636b04-7m58c\" (UID: \"a9e3bd68-a39f-49ad-8ab2-4baa27988e22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-636b04-7m58c"
Sep 06 10:45:32.323118 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:32.323085    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9e3bd68-a39f-49ad-8ab2-4baa27988e22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-636b04-7m58c\" (UID: \"a9e3bd68-a39f-49ad-8ab2-4baa27988e22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-636b04-7m58c"
Sep 06 10:45:32.323704 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:32.323685    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9e3bd68-a39f-49ad-8ab2-4baa27988e22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-636b04-7m58c\" (UID: \"a9e3bd68-a39f-49ad-8ab2-4baa27988e22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-636b04-7m58c"
Sep 06 10:45:32.443230 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:32.443137    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-636b04-7m58c"
Sep 06 10:45:32.713959 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:32.713930    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-636b04-7m58c"]
Sep 06 10:45:32.716427 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:45:32.716398    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9e3bd68_a39f_49ad_8ab2_4baa27988e22.slice/crio-1405f135c08b28387da22dd6c3c6211d16fdcb676c53b27c132b741bb52734d8 WatchSource:0}: Error finding container 1405f135c08b28387da22dd6c3c6211d16fdcb676c53b27c132b741bb52734d8: Status 404 returned error can't find the container with id 1405f135c08b28387da22dd6c3c6211d16fdcb676c53b27c132b741bb52734d8
Sep 06 10:45:33.018384 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:33.018349    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-636b04-7m58c" event={"ID":"a9e3bd68-a39f-49ad-8ab2-4baa27988e22","Type":"ContainerStarted","Data":"98c8479fea72f8fe7b57bf8cb77ea21943abebb7f23981d3b14e2ffe59caed44"}
Sep 06 10:45:33.018614 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:33.018408    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-636b04-7m58c" event={"ID":"a9e3bd68-a39f-49ad-8ab2-4baa27988e22","Type":"ContainerStarted","Data":"1405f135c08b28387da22dd6c3c6211d16fdcb676c53b27c132b741bb52734d8"}
Sep 06 10:45:33.034069 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:33.034022    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-636b04-7m58c" podStartSLOduration=1.034006626 podStartE2EDuration="1.034006626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:45:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:45:33.032026832 +0000 UTC m=+945.481807601" watchObservedRunningTime="2026-09-06 10:45:33.034006626 +0000 UTC m=+945.483787424"
Sep 06 10:45:34.025059 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:34.025024    2557 generic.go:356] "Generic (PLEG): container finished" podID="a9e3bd68-a39f-49ad-8ab2-4baa27988e22" containerID="98c8479fea72f8fe7b57bf8cb77ea21943abebb7f23981d3b14e2ffe59caed44" exitCode=0
Sep 06 10:45:34.025524 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:34.025113    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-636b04-7m58c" event={"ID":"a9e3bd68-a39f-49ad-8ab2-4baa27988e22","Type":"ContainerDied","Data":"98c8479fea72f8fe7b57bf8cb77ea21943abebb7f23981d3b14e2ffe59caed44"}
Sep 06 10:45:34.967671 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:34.967631    2557 patch_prober.go:28] interesting pod/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 06 10:45:34.967865 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:34.967703    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k" podUID="df0c73e7-c6e2-47b5-a2b9-daaaed78af36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 06 10:45:35.169502 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:35.169468    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-636b04-7m58c"
Sep 06 10:45:35.250908 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:35.250878    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9e3bd68-a39f-49ad-8ab2-4baa27988e22-maas-api-service-ca\") pod \"a9e3bd68-a39f-49ad-8ab2-4baa27988e22\" (UID: \"a9e3bd68-a39f-49ad-8ab2-4baa27988e22\") "
Sep 06 10:45:35.251424 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:35.251239    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9e3bd68-a39f-49ad-8ab2-4baa27988e22-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a9e3bd68-a39f-49ad-8ab2-4baa27988e22" (UID: "a9e3bd68-a39f-49ad-8ab2-4baa27988e22"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:45:35.351816 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:35.351787    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9e3bd68-a39f-49ad-8ab2-4baa27988e22-maas-api-service-ca\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:36.039294 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:36.039264    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-636b04-7m58c"
Sep 06 10:45:36.039294 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:36.039281    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-636b04-7m58c" event={"ID":"a9e3bd68-a39f-49ad-8ab2-4baa27988e22","Type":"ContainerDied","Data":"1405f135c08b28387da22dd6c3c6211d16fdcb676c53b27c132b741bb52734d8"}
Sep 06 10:45:36.039549 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:36.039336    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1405f135c08b28387da22dd6c3c6211d16fdcb676c53b27c132b741bb52734d8"
Sep 06 10:45:36.041133 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:36.041110    2557 generic.go:356] "Generic (PLEG): container finished" podID="df0c73e7-c6e2-47b5-a2b9-daaaed78af36" containerID="1b4f8f1501220fd735928c98a575b617eeebc238a84e8085dffb11f0eea84522" exitCode=0
Sep 06 10:45:36.041261 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:36.041187    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k" event={"ID":"df0c73e7-c6e2-47b5-a2b9-daaaed78af36","Type":"ContainerDied","Data":"1b4f8f1501220fd735928c98a575b617eeebc238a84e8085dffb11f0eea84522"}
Sep 06 10:45:36.041667 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:36.041651    2557 scope.go:117] "RemoveContainer" containerID="1b4f8f1501220fd735928c98a575b617eeebc238a84e8085dffb11f0eea84522"
Sep 06 10:45:36.042751 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:36.042723    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:45:36.042810 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:36.042792    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:45:36.042846 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:36.042822    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:45:37.049299 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:37.049267    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k" event={"ID":"df0c73e7-c6e2-47b5-a2b9-daaaed78af36","Type":"ContainerStarted","Data":"0403895c3bb30ad78ae0eeeed60d7eec0d7665d57f92b9a9609ea1353a391193"}
Sep 06 10:45:37.635324 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:37.635284    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:37.635556 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:37.635517    2557 patch_prober.go:28] interesting pod/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 06 10:45:37.635652 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:37.635571    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k" podUID="df0c73e7-c6e2-47b5-a2b9-daaaed78af36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 06 10:45:38.636211 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:38.636173    2557 patch_prober.go:28] interesting pod/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 06 10:45:38.636614 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:38.636236    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k" podUID="df0c73e7-c6e2-47b5-a2b9-daaaed78af36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 06 10:45:39.635327 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:39.635290    2557 patch_prober.go:28] interesting pod/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 06 10:45:39.635534 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:39.635348    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k" podUID="df0c73e7-c6e2-47b5-a2b9-daaaed78af36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 06 10:45:40.635814 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:40.635778    2557 patch_prober.go:28] interesting pod/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 06 10:45:40.636216 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:40.635834    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k" podUID="df0c73e7-c6e2-47b5-a2b9-daaaed78af36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 06 10:45:41.635838 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:41.635807    2557 patch_prober.go:28] interesting pod/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 06 10:45:41.636207 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:41.635857    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k" podUID="df0c73e7-c6e2-47b5-a2b9-daaaed78af36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 06 10:45:42.481569 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:42.481516    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-636b04-29811525-jmhfd"]
Sep 06 10:45:42.483706 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:42.483681    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-636b04-29811525-jmhfd"]
Sep 06 10:45:42.638857 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:42.638828    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:42.639220 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:42.639076    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:42.639824 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:42.639804    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:45:42.699480 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:42.699446    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-636b04-7m58c"]
Sep 06 10:45:42.702390 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:42.702366    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-636b04-7m58c"]
Sep 06 10:45:44.037282 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:44.037247    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9e3bd68-a39f-49ad-8ab2-4baa27988e22" path="/var/lib/kubelet/pods/a9e3bd68-a39f-49ad-8ab2-4baa27988e22/volumes"
Sep 06 10:45:44.037637 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:44.037562    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7b102fb-7829-4eff-a3e5-55facbb633ca" path="/var/lib/kubelet/pods/b7b102fb-7829-4eff-a3e5-55facbb633ca/volumes"
Sep 06 10:45:46.111307 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:46.111263    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:45:49.072480 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:49.072446    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"]
Sep 06 10:45:49.073277 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:49.073231    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z" podUID="5dc1d481-5896-4ce2-8360-ab174e11945a" containerName="istio-proxy" containerID="cri-o://feccd848d63f7185a2a6181b8c8dcc6b581254c6febebdcf9348e3f406b26318" gracePeriod=30
Sep 06 10:45:54.331669 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.331643    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:45:54.421834 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.421742    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-podinfo\") pod \"5dc1d481-5896-4ce2-8360-ab174e11945a\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") "
Sep 06 10:45:54.421834 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.421794    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-envoy\") pod \"5dc1d481-5896-4ce2-8360-ab174e11945a\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") "
Sep 06 10:45:54.422059 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.421851    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d55r4\" (UniqueName: \"kubernetes.io/projected/5dc1d481-5896-4ce2-8360-ab174e11945a-kube-api-access-d55r4\") pod \"5dc1d481-5896-4ce2-8360-ab174e11945a\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") "
Sep 06 10:45:54.422059 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.421876    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-credential-socket\") pod \"5dc1d481-5896-4ce2-8360-ab174e11945a\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") "
Sep 06 10:45:54.422059 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.421901    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-workload-certs\") pod \"5dc1d481-5896-4ce2-8360-ab174e11945a\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") "
Sep 06 10:45:54.422059 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.421930    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-workload-socket\") pod \"5dc1d481-5896-4ce2-8360-ab174e11945a\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") "
Sep 06 10:45:54.422059 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.421956    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-token\") pod \"5dc1d481-5896-4ce2-8360-ab174e11945a\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") "
Sep 06 10:45:54.422059 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.422029    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dc1d481-5896-4ce2-8360-ab174e11945a-istiod-ca-cert\") pod \"5dc1d481-5896-4ce2-8360-ab174e11945a\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") "
Sep 06 10:45:54.422425 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.422099    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-data\") pod \"5dc1d481-5896-4ce2-8360-ab174e11945a\" (UID: \"5dc1d481-5896-4ce2-8360-ab174e11945a\") "
Sep 06 10:45:54.422425 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.422247    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5dc1d481-5896-4ce2-8360-ab174e11945a" (UID: "5dc1d481-5896-4ce2-8360-ab174e11945a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:45:54.422425 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.422285    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5dc1d481-5896-4ce2-8360-ab174e11945a" (UID: "5dc1d481-5896-4ce2-8360-ab174e11945a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:45:54.422607 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.422472    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-data" (OuterVolumeSpecName: "istio-data") pod "5dc1d481-5896-4ce2-8360-ab174e11945a" (UID: "5dc1d481-5896-4ce2-8360-ab174e11945a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:45:54.422607 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.422536    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-credential-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:54.422607 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.422558    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-workload-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:54.422764 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.422711    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dc1d481-5896-4ce2-8360-ab174e11945a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5dc1d481-5896-4ce2-8360-ab174e11945a" (UID: "5dc1d481-5896-4ce2-8360-ab174e11945a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:45:54.422920 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.422882    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5dc1d481-5896-4ce2-8360-ab174e11945a" (UID: "5dc1d481-5896-4ce2-8360-ab174e11945a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:45:54.424245 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.424222    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dc1d481-5896-4ce2-8360-ab174e11945a-kube-api-access-d55r4" (OuterVolumeSpecName: "kube-api-access-d55r4") pod "5dc1d481-5896-4ce2-8360-ab174e11945a" (UID: "5dc1d481-5896-4ce2-8360-ab174e11945a"). InnerVolumeSpecName "kube-api-access-d55r4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:45:54.424367 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.424347    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5dc1d481-5896-4ce2-8360-ab174e11945a" (UID: "5dc1d481-5896-4ce2-8360-ab174e11945a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 10:45:54.424527 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.424500    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5dc1d481-5896-4ce2-8360-ab174e11945a" (UID: "5dc1d481-5896-4ce2-8360-ab174e11945a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:45:54.424942 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.424917    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-token" (OuterVolumeSpecName: "istio-token") pod "5dc1d481-5896-4ce2-8360-ab174e11945a" (UID: "5dc1d481-5896-4ce2-8360-ab174e11945a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:45:54.523676 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.523643    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-data\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:54.523676 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.523670    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-podinfo\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:54.523676 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.523681    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-envoy\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:54.523980 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.523691    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d55r4\" (UniqueName: \"kubernetes.io/projected/5dc1d481-5896-4ce2-8360-ab174e11945a-kube-api-access-d55r4\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:54.523980 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.523704    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dc1d481-5896-4ce2-8360-ab174e11945a-workload-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:54.523980 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.523717    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dc1d481-5896-4ce2-8360-ab174e11945a-istio-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:54.523980 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:54.523725    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dc1d481-5896-4ce2-8360-ab174e11945a-istiod-ca-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:45:55.154485 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:55.154451    2557 generic.go:356] "Generic (PLEG): container finished" podID="5dc1d481-5896-4ce2-8360-ab174e11945a" containerID="feccd848d63f7185a2a6181b8c8dcc6b581254c6febebdcf9348e3f406b26318" exitCode=0
Sep 06 10:45:55.154680 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:55.154526    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"
Sep 06 10:45:55.154680 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:55.154536    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z" event={"ID":"5dc1d481-5896-4ce2-8360-ab174e11945a","Type":"ContainerDied","Data":"feccd848d63f7185a2a6181b8c8dcc6b581254c6febebdcf9348e3f406b26318"}
Sep 06 10:45:55.154680 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:55.154600    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z" event={"ID":"5dc1d481-5896-4ce2-8360-ab174e11945a","Type":"ContainerDied","Data":"719ca724b7dfd5cbcc2316762af553a052afc43eb226dffa506550d72dc64e66"}
Sep 06 10:45:55.154680 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:55.154623    2557 scope.go:117] "RemoveContainer" containerID="feccd848d63f7185a2a6181b8c8dcc6b581254c6febebdcf9348e3f406b26318"
Sep 06 10:45:55.164734 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:55.164717    2557 scope.go:117] "RemoveContainer" containerID="3d3828292538bf177f5f5e75ebc026253c5d9c2dcf02184705e5a3e606a8ed04"
Sep 06 10:45:55.173722 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:55.173705    2557 scope.go:117] "RemoveContainer" containerID="feccd848d63f7185a2a6181b8c8dcc6b581254c6febebdcf9348e3f406b26318"
Sep 06 10:45:55.174051 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:45:55.174025    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feccd848d63f7185a2a6181b8c8dcc6b581254c6febebdcf9348e3f406b26318\": container with ID starting with feccd848d63f7185a2a6181b8c8dcc6b581254c6febebdcf9348e3f406b26318 not found: ID does not exist" containerID="feccd848d63f7185a2a6181b8c8dcc6b581254c6febebdcf9348e3f406b26318"
Sep 06 10:45:55.174181 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:55.174057    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feccd848d63f7185a2a6181b8c8dcc6b581254c6febebdcf9348e3f406b26318"} err="failed to get container status \"feccd848d63f7185a2a6181b8c8dcc6b581254c6febebdcf9348e3f406b26318\": rpc error: code = NotFound desc = could not find container \"feccd848d63f7185a2a6181b8c8dcc6b581254c6febebdcf9348e3f406b26318\": container with ID starting with feccd848d63f7185a2a6181b8c8dcc6b581254c6febebdcf9348e3f406b26318 not found: ID does not exist"
Sep 06 10:45:55.174181 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:55.174077    2557 scope.go:117] "RemoveContainer" containerID="3d3828292538bf177f5f5e75ebc026253c5d9c2dcf02184705e5a3e606a8ed04"
Sep 06 10:45:55.174385 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:45:55.174366    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d3828292538bf177f5f5e75ebc026253c5d9c2dcf02184705e5a3e606a8ed04\": container with ID starting with 3d3828292538bf177f5f5e75ebc026253c5d9c2dcf02184705e5a3e606a8ed04 not found: ID does not exist" containerID="3d3828292538bf177f5f5e75ebc026253c5d9c2dcf02184705e5a3e606a8ed04"
Sep 06 10:45:55.174447 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:55.174391    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d3828292538bf177f5f5e75ebc026253c5d9c2dcf02184705e5a3e606a8ed04"} err="failed to get container status \"3d3828292538bf177f5f5e75ebc026253c5d9c2dcf02184705e5a3e606a8ed04\": rpc error: code = NotFound desc = could not find container \"3d3828292538bf177f5f5e75ebc026253c5d9c2dcf02184705e5a3e606a8ed04\": container with ID starting with 3d3828292538bf177f5f5e75ebc026253c5d9c2dcf02184705e5a3e606a8ed04 not found: ID does not exist"
Sep 06 10:45:55.175690 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:55.175670    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"]
Sep 06 10:45:55.179114 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:55.179094    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-636b04-openshift-default-85cd759bc4-9652z"]
Sep 06 10:45:56.036713 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:56.036677    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dc1d481-5896-4ce2-8360-ab174e11945a" path="/var/lib/kubelet/pods/5dc1d481-5896-4ce2-8360-ab174e11945a/volumes"
Sep 06 10:45:59.119018 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:45:59.118981    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:46:06.716944 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:06.716903    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:46:09.018299 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:09.018257    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"]
Sep 06 10:46:09.018723 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:09.018473    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k" podUID="df0c73e7-c6e2-47b5-a2b9-daaaed78af36" containerName="istio-proxy" containerID="cri-o://0403895c3bb30ad78ae0eeeed60d7eec0d7665d57f92b9a9609ea1353a391193" gracePeriod=30
Sep 06 10:46:12.641465 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:12.641374    2557 patch_prober.go:28] interesting pod/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 06 10:46:12.641465 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:12.641447    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k" podUID="df0c73e7-c6e2-47b5-a2b9-daaaed78af36" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 10:46:14.267212 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.267183    2557 generic.go:356] "Generic (PLEG): container finished" podID="df0c73e7-c6e2-47b5-a2b9-daaaed78af36" containerID="0403895c3bb30ad78ae0eeeed60d7eec0d7665d57f92b9a9609ea1353a391193" exitCode=0
Sep 06 10:46:14.267617 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.267267    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k" event={"ID":"df0c73e7-c6e2-47b5-a2b9-daaaed78af36","Type":"ContainerDied","Data":"0403895c3bb30ad78ae0eeeed60d7eec0d7665d57f92b9a9609ea1353a391193"}
Sep 06 10:46:14.267617 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.267345    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k" event={"ID":"df0c73e7-c6e2-47b5-a2b9-daaaed78af36","Type":"ContainerDied","Data":"22f194b2a24572550dddd5590ebda0508a2c9f8b0f2cb7e0939858df39614d46"}
Sep 06 10:46:14.267617 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.267371    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22f194b2a24572550dddd5590ebda0508a2c9f8b0f2cb7e0939858df39614d46"
Sep 06 10:46:14.267617 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.267392    2557 scope.go:117] "RemoveContainer" containerID="1b4f8f1501220fd735928c98a575b617eeebc238a84e8085dffb11f0eea84522"
Sep 06 10:46:14.272817 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.272797    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:46:14.402120 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.402091    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-podinfo\") pod \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") "
Sep 06 10:46:14.402277 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.402136    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-envoy\") pod \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") "
Sep 06 10:46:14.402277 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.402168    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzlxg\" (UniqueName: \"kubernetes.io/projected/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-kube-api-access-vzlxg\") pod \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") "
Sep 06 10:46:14.402277 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.402195    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-credential-socket\") pod \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") "
Sep 06 10:46:14.402277 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.402266    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-data\") pod \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") "
Sep 06 10:46:14.402484 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.402297    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-workload-socket\") pod \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") "
Sep 06 10:46:14.402484 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.402333    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istiod-ca-cert\") pod \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") "
Sep 06 10:46:14.402484 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.402380    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-workload-certs\") pod \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") "
Sep 06 10:46:14.402484 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.402410    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-token\") pod \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\" (UID: \"df0c73e7-c6e2-47b5-a2b9-daaaed78af36\") "
Sep 06 10:46:14.402732 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.402591    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "df0c73e7-c6e2-47b5-a2b9-daaaed78af36" (UID: "df0c73e7-c6e2-47b5-a2b9-daaaed78af36"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:46:14.402732 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.402628    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-data" (OuterVolumeSpecName: "istio-data") pod "df0c73e7-c6e2-47b5-a2b9-daaaed78af36" (UID: "df0c73e7-c6e2-47b5-a2b9-daaaed78af36"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:46:14.402732 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.402677    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "df0c73e7-c6e2-47b5-a2b9-daaaed78af36" (UID: "df0c73e7-c6e2-47b5-a2b9-daaaed78af36"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:46:14.402900 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.402744    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-credential-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:46:14.402900 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.402763    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-data\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:46:14.403010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.402942    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "df0c73e7-c6e2-47b5-a2b9-daaaed78af36" (UID: "df0c73e7-c6e2-47b5-a2b9-daaaed78af36"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:46:14.403010 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.402947    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "df0c73e7-c6e2-47b5-a2b9-daaaed78af36" (UID: "df0c73e7-c6e2-47b5-a2b9-daaaed78af36"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:46:14.404365 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.404337    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "df0c73e7-c6e2-47b5-a2b9-daaaed78af36" (UID: "df0c73e7-c6e2-47b5-a2b9-daaaed78af36"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 10:46:14.404499 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.404445    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-token" (OuterVolumeSpecName: "istio-token") pod "df0c73e7-c6e2-47b5-a2b9-daaaed78af36" (UID: "df0c73e7-c6e2-47b5-a2b9-daaaed78af36"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:46:14.404776 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.404754    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "df0c73e7-c6e2-47b5-a2b9-daaaed78af36" (UID: "df0c73e7-c6e2-47b5-a2b9-daaaed78af36"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:46:14.404776 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.404770    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-kube-api-access-vzlxg" (OuterVolumeSpecName: "kube-api-access-vzlxg") pod "df0c73e7-c6e2-47b5-a2b9-daaaed78af36" (UID: "df0c73e7-c6e2-47b5-a2b9-daaaed78af36"). InnerVolumeSpecName "kube-api-access-vzlxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:46:14.503600 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.503556    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-workload-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:46:14.503600 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.503595    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istiod-ca-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:46:14.503600 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.503605    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-workload-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:46:14.503600 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.503612    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:46:14.503600 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.503620    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-podinfo\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:46:14.503897 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.503627    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-istio-envoy\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:46:14.503897 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:14.503636    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzlxg\" (UniqueName: \"kubernetes.io/projected/df0c73e7-c6e2-47b5-a2b9-daaaed78af36-kube-api-access-vzlxg\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:46:15.274449 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:15.274415    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"
Sep 06 10:46:15.307427 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:15.307395    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"]
Sep 06 10:46:15.311089 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:15.311065    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2313d276-openshift-default-77b87b556d-tsh7k"]
Sep 06 10:46:16.036392 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:16.036355    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df0c73e7-c6e2-47b5-a2b9-daaaed78af36" path="/var/lib/kubelet/pods/df0c73e7-c6e2-47b5-a2b9-daaaed78af36/volumes"
Sep 06 10:46:18.372133 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.372102    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"]
Sep 06 10:46:18.372614 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.372532    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df0c73e7-c6e2-47b5-a2b9-daaaed78af36" containerName="istio-proxy"
Sep 06 10:46:18.372614 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.372546    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="df0c73e7-c6e2-47b5-a2b9-daaaed78af36" containerName="istio-proxy"
Sep 06 10:46:18.372614 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.372554    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dc1d481-5896-4ce2-8360-ab174e11945a" containerName="istio-proxy"
Sep 06 10:46:18.372614 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.372559    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dc1d481-5896-4ce2-8360-ab174e11945a" containerName="istio-proxy"
Sep 06 10:46:18.372614 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.372571    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df0c73e7-c6e2-47b5-a2b9-daaaed78af36" containerName="istio-proxy"
Sep 06 10:46:18.372614 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.372578    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="df0c73e7-c6e2-47b5-a2b9-daaaed78af36" containerName="istio-proxy"
Sep 06 10:46:18.372614 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.372601    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dc1d481-5896-4ce2-8360-ab174e11945a" containerName="istio-proxy"
Sep 06 10:46:18.372614 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.372610    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dc1d481-5896-4ce2-8360-ab174e11945a" containerName="istio-proxy"
Sep 06 10:46:18.373022 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.372631    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9e3bd68-a39f-49ad-8ab2-4baa27988e22" containerName="revoke-keys"
Sep 06 10:46:18.373022 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.372639    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9e3bd68-a39f-49ad-8ab2-4baa27988e22" containerName="revoke-keys"
Sep 06 10:46:18.373022 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.372725    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dc1d481-5896-4ce2-8360-ab174e11945a" containerName="istio-proxy"
Sep 06 10:46:18.373022 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.372741    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9e3bd68-a39f-49ad-8ab2-4baa27988e22" containerName="revoke-keys"
Sep 06 10:46:18.373022 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.372751    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="df0c73e7-c6e2-47b5-a2b9-daaaed78af36" containerName="istio-proxy"
Sep 06 10:46:18.373022 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.372757    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dc1d481-5896-4ce2-8360-ab174e11945a" containerName="istio-proxy"
Sep 06 10:46:18.373022 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.372764    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="df0c73e7-c6e2-47b5-a2b9-daaaed78af36" containerName="istio-proxy"
Sep 06 10:46:18.374786 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.374771    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.377458 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.377434    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-eb4b45-openshift-default-dockercfg-x2gfh\""
Sep 06 10:46:18.387053 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.387027    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"]
Sep 06 10:46:18.440946 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.440919    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istiod-ca-cert\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.441089 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.440950    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-podinfo\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.441089 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.440993    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-token\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.441089 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.441074    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-workload-socket\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.441219 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.441110    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-data\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.441256 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.441220    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-workload-certs\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.441297 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.441265    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2swtt\" (UniqueName: \"kubernetes.io/projected/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-kube-api-access-2swtt\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.441331 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.441294    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-envoy\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.441331 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.441321    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-credential-socket\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.542114 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.542074    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-workload-certs\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.542114 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.542118    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2swtt\" (UniqueName: \"kubernetes.io/projected/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-kube-api-access-2swtt\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.542380 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.542138    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-envoy\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.542380 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.542251    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-credential-socket\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.542380 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.542336    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istiod-ca-cert\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.542380 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.542363    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-podinfo\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.542602 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.542418    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-token\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.542602 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.542477    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-workload-socket\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.542602 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.542516    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-data\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.542602 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.542577    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-workload-certs\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.542804 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.542621    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-credential-socket\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.542804 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.542778    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-data\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.542910 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.542850    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-workload-socket\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.543232 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.543206    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istiod-ca-cert\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.544361 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.544336    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-envoy\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.544891 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.544874    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-podinfo\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.549818 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.549797    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-token\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.550598 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.550583    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2swtt\" (UniqueName: \"kubernetes.io/projected/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-kube-api-access-2swtt\") pod \"e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") " pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.688389 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.688291    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:18.962131 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.962101    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"]
Sep 06 10:46:18.964188 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:46:18.964160    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd66f3fa_b46c_4850_9f62_c97c1a9fa32c.slice/crio-11ce0f97b2974f443a21e4cc85bedb97e2be7a5072bef83732542e548b019b23 WatchSource:0}: Error finding container 11ce0f97b2974f443a21e4cc85bedb97e2be7a5072bef83732542e548b019b23: Status 404 returned error can't find the container with id 11ce0f97b2974f443a21e4cc85bedb97e2be7a5072bef83732542e548b019b23
Sep 06 10:46:18.967015 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.966974    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:46:18.967118 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.967058    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:46:18.967118 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:18.967099    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:46:19.298245 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:19.298145    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs" event={"ID":"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c","Type":"ContainerStarted","Data":"5cf684e5812f4beb7e98d8266f5641b0ef1ac6d08fa7d482b9a4e9e83f7ec762"}
Sep 06 10:46:19.298245 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:19.298215    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs" event={"ID":"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c","Type":"ContainerStarted","Data":"11ce0f97b2974f443a21e4cc85bedb97e2be7a5072bef83732542e548b019b23"}
Sep 06 10:46:19.317651 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:19.317600    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs" podStartSLOduration=1.317585788 podStartE2EDuration="1.317585788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:46:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:46:19.314771271 +0000 UTC m=+991.764552045" watchObservedRunningTime="2026-09-06 10:46:19.317585788 +0000 UTC m=+991.767366556"
Sep 06 10:46:19.689070 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:19.689032    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:20.694032 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:20.693968    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:21.310554 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:21.310521    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:21.311745 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:21.311721    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:22.490784 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.490750    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"]
Sep 06 10:46:22.493969 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.493950    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.496271 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.496245    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-72cb16-openshift-default-dockercfg-khpvh\""
Sep 06 10:46:22.505740 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.505717    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"]
Sep 06 10:46:22.581720 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.581683    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-istio-envoy\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.581894 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.581744    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-istio-data\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.581894 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.581876    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/475cab92-806a-4de2-bc9b-2b33485eba74-istio-podinfo\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.582026 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.581920    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x7sl\" (UniqueName: \"kubernetes.io/projected/475cab92-806a-4de2-bc9b-2b33485eba74-kube-api-access-8x7sl\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.582026 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.582004    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-workload-socket\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.582140 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.582059    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/475cab92-806a-4de2-bc9b-2b33485eba74-istio-token\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.582140 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.582081    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-workload-certs\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.582140 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.582102    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/475cab92-806a-4de2-bc9b-2b33485eba74-istiod-ca-cert\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.582300 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.582185    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-credential-socket\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.683046 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.683008    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/475cab92-806a-4de2-bc9b-2b33485eba74-istio-token\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.683046 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.683050    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-workload-certs\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.683280 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.683074    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/475cab92-806a-4de2-bc9b-2b33485eba74-istiod-ca-cert\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.683280 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.683119    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-credential-socket\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.683280 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.683238    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-istio-envoy\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.683456 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.683308    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-istio-data\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.683456 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.683396    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/475cab92-806a-4de2-bc9b-2b33485eba74-istio-podinfo\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.683456 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.683429    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x7sl\" (UniqueName: \"kubernetes.io/projected/475cab92-806a-4de2-bc9b-2b33485eba74-kube-api-access-8x7sl\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.683650 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.683517    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-workload-socket\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.683650 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.683547    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-credential-socket\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.683764 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.683701    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-workload-certs\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.683874 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.683856    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-istio-data\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.683961 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.683941    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-workload-socket\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.684111 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.684093    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/475cab92-806a-4de2-bc9b-2b33485eba74-istiod-ca-cert\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.685916 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.685897    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-istio-envoy\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.686095 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.686072    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/475cab92-806a-4de2-bc9b-2b33485eba74-istio-podinfo\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.694468 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.694444    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/475cab92-806a-4de2-bc9b-2b33485eba74-istio-token\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.694870 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.694852    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x7sl\" (UniqueName: \"kubernetes.io/projected/475cab92-806a-4de2-bc9b-2b33485eba74-kube-api-access-8x7sl\") pod \"e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") " pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:22.808142 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:22.808062    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:23.084279 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:23.084254    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"]
Sep 06 10:46:23.087333 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:46:23.087305    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod475cab92_806a_4de2_bc9b_2b33485eba74.slice/crio-88c7acd7f01f6f702096c808b488b63ad67aa22d68b70a0629067c0c9d96775d WatchSource:0}: Error finding container 88c7acd7f01f6f702096c808b488b63ad67aa22d68b70a0629067c0c9d96775d: Status 404 returned error can't find the container with id 88c7acd7f01f6f702096c808b488b63ad67aa22d68b70a0629067c0c9d96775d
Sep 06 10:46:23.089682 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:23.089650    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:46:23.089754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:23.089715    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:46:23.089754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:23.089744    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:46:23.323535 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:23.323476    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5" event={"ID":"475cab92-806a-4de2-bc9b-2b33485eba74","Type":"ContainerStarted","Data":"d21dba38d774e7059070945a5b9ce8bb77a3192122705baa7370c772a572e401"}
Sep 06 10:46:23.323690 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:23.323560    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5" event={"ID":"475cab92-806a-4de2-bc9b-2b33485eba74","Type":"ContainerStarted","Data":"88c7acd7f01f6f702096c808b488b63ad67aa22d68b70a0629067c0c9d96775d"}
Sep 06 10:46:23.348796 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:23.348752    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5" podStartSLOduration=1.348735394 podStartE2EDuration="1.348735394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:46:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:46:23.346519947 +0000 UTC m=+995.796300717" watchObservedRunningTime="2026-09-06 10:46:23.348735394 +0000 UTC m=+995.798516163"
Sep 06 10:46:23.808868 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:23.808780    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:24.813247 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:24.813212    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:25.335912 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:25.335877    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:25.336939 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:25.336909    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:26.021776 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:26.021744    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:46:36.889083 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:36.889046    2557 patch_prober.go:28] interesting pod/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": read tcp 10.132.0.2:34594->10.132.0.64:15021: read: connection reset by peer" start-of-body=
Sep 06 10:46:36.889458 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:36.889134    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs" podUID="dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": read tcp 10.132.0.2:34594->10.132.0.64:15021: read: connection reset by peer"
Sep 06 10:46:37.418073 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:37.418040    2557 generic.go:356] "Generic (PLEG): container finished" podID="dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" containerID="5cf684e5812f4beb7e98d8266f5641b0ef1ac6d08fa7d482b9a4e9e83f7ec762" exitCode=0
Sep 06 10:46:37.418251 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:37.418112    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs" event={"ID":"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c","Type":"ContainerDied","Data":"5cf684e5812f4beb7e98d8266f5641b0ef1ac6d08fa7d482b9a4e9e83f7ec762"}
Sep 06 10:46:37.418550 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:37.418536    2557 scope.go:117] "RemoveContainer" containerID="5cf684e5812f4beb7e98d8266f5641b0ef1ac6d08fa7d482b9a4e9e83f7ec762"
Sep 06 10:46:37.419696 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:37.419668    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:46:37.419760 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:37.419739    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:46:37.419797 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:37.419774    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:46:38.425873 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:38.425839    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs" event={"ID":"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c","Type":"ContainerStarted","Data":"d77b15da80019fd6a04e3dd11e7dff7febb12d99c0e0ed8d618f7519e0bf6e66"}
Sep 06 10:46:38.688778 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:38.688670    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:38.688985 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:38.688959    2557 patch_prober.go:28] interesting pod/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 06 10:46:38.689045 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:38.689021    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs" podUID="dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 06 10:46:38.724329 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:38.724295    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:46:39.689226 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:39.689179    2557 patch_prober.go:28] interesting pod/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 06 10:46:39.689743 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:39.689251    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs" podUID="dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 06 10:46:40.689549 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:40.689482    2557 patch_prober.go:28] interesting pod/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 06 10:46:40.690057 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:40.689610    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs" podUID="dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 06 10:46:41.689249 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:41.689200    2557 patch_prober.go:28] interesting pod/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 06 10:46:41.689464 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:41.689273    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs" podUID="dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 06 10:46:42.689029 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:42.688982    2557 patch_prober.go:28] interesting pod/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 06 10:46:42.689556 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:42.689053    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs" podUID="dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 06 10:46:43.688927 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:43.688887    2557 patch_prober.go:28] interesting pod/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 06 10:46:43.689239 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:43.689207    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs" podUID="dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 06 10:46:44.693239 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:44.693196    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:44.693697 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:44.693580    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:44.694374 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:44.694356    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:46:48.486573 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:48.486454    2557 generic.go:356] "Generic (PLEG): container finished" podID="475cab92-806a-4de2-bc9b-2b33485eba74" containerID="d21dba38d774e7059070945a5b9ce8bb77a3192122705baa7370c772a572e401" exitCode=0
Sep 06 10:46:48.486573 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:48.486528    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5" event={"ID":"475cab92-806a-4de2-bc9b-2b33485eba74","Type":"ContainerDied","Data":"d21dba38d774e7059070945a5b9ce8bb77a3192122705baa7370c772a572e401"}
Sep 06 10:46:48.486983 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:48.486967    2557 scope.go:117] "RemoveContainer" containerID="d21dba38d774e7059070945a5b9ce8bb77a3192122705baa7370c772a572e401"
Sep 06 10:46:48.488108 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:48.488076    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:46:48.488198 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:48.488159    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:46:48.488198 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:48.488190    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:46:49.498087 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:49.498052    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5" event={"ID":"475cab92-806a-4de2-bc9b-2b33485eba74","Type":"ContainerStarted","Data":"101e19f5911f1e5335d4e598b4914473a510b34888d02e764a0c291f46b25a42"}
Sep 06 10:46:49.808865 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:49.808785    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:49.809017 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:49.808971    2557 patch_prober.go:28] interesting pod/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 06 10:46:49.809065 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:49.809034    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5" podUID="475cab92-806a-4de2-bc9b-2b33485eba74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 06 10:46:50.809124 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:50.809086    2557 patch_prober.go:28] interesting pod/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 06 10:46:50.809573 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:50.809164    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5" podUID="475cab92-806a-4de2-bc9b-2b33485eba74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 06 10:46:50.829659 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:50.829628    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"]
Sep 06 10:46:50.833943 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:50.833922    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:50.836237 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:50.836211    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-fbf820-openshift-default-dockercfg-ttzb7\""
Sep 06 10:46:50.842992 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:50.842879    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"]
Sep 06 10:46:50.993583 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:50.989729    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-envoy\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:50.993790 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:50.993663    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rhmv\" (UniqueName: \"kubernetes.io/projected/3b061474-63c2-4ca7-b309-189cb3bad4d7-kube-api-access-5rhmv\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:50.993790 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:50.993714    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-podinfo\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:50.993790 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:50.993765    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b061474-63c2-4ca7-b309-189cb3bad4d7-istiod-ca-cert\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:50.993966 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:50.993795    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-data\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:50.993966 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:50.993881    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-token\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:50.993966 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:50.993941    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-workload-socket\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:50.994129 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:50.994019    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-credential-socket\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:50.994129 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:50.994117    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-workload-certs\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.095474 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.095427    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-workload-socket\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.095669 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.095529    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-credential-socket\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.095669 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.095581    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-workload-certs\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.095669 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.095616    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-envoy\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.095669 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.095642    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rhmv\" (UniqueName: \"kubernetes.io/projected/3b061474-63c2-4ca7-b309-189cb3bad4d7-kube-api-access-5rhmv\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.095889 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.095671    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-podinfo\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.095889 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.095710    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b061474-63c2-4ca7-b309-189cb3bad4d7-istiod-ca-cert\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.095889 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.095736    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-data\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.095889 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.095797    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-token\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.096097 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.095915    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-workload-socket\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.096097 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.095958    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-credential-socket\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.096097 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.095986    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-workload-certs\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.096276 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.096254    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-data\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.096565 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.096540    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b061474-63c2-4ca7-b309-189cb3bad4d7-istiod-ca-cert\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.098024 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.097997    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-envoy\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.098316 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.098300    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-podinfo\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.103142 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.103114    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-token\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.103225 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.103148    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rhmv\" (UniqueName: \"kubernetes.io/projected/3b061474-63c2-4ca7-b309-189cb3bad4d7-kube-api-access-5rhmv\") pod \"e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") " pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.147195 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.147166    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:51.448221 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.448197    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"]
Sep 06 10:46:51.450116 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:46:51.450089    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b061474_63c2_4ca7_b309_189cb3bad4d7.slice/crio-7518a5df0e6d6d2fb330acfb8b259379a661c4530b368d7662ca97fa134a70ab WatchSource:0}: Error finding container 7518a5df0e6d6d2fb330acfb8b259379a661c4530b368d7662ca97fa134a70ab: Status 404 returned error can't find the container with id 7518a5df0e6d6d2fb330acfb8b259379a661c4530b368d7662ca97fa134a70ab
Sep 06 10:46:51.452320 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.452290    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:46:51.452393 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.452354    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:46:51.452393 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.452382    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:46:51.510750 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.510716    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff" event={"ID":"3b061474-63c2-4ca7-b309-189cb3bad4d7","Type":"ContainerStarted","Data":"7518a5df0e6d6d2fb330acfb8b259379a661c4530b368d7662ca97fa134a70ab"}
Sep 06 10:46:51.809422 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.809345    2557 patch_prober.go:28] interesting pod/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 06 10:46:51.809422 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:51.809398    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5" podUID="475cab92-806a-4de2-bc9b-2b33485eba74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 06 10:46:52.518233 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:52.518197    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff" event={"ID":"3b061474-63c2-4ca7-b309-189cb3bad4d7","Type":"ContainerStarted","Data":"dc8a68620f3e1b12c5e3fa266418e280d069bb9de89bf88c93550f5f4d81da2c"}
Sep 06 10:46:52.538804 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:52.538753    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff" podStartSLOduration=2.538739562 podStartE2EDuration="2.538739562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:46:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:46:52.536948406 +0000 UTC m=+1024.986729174" watchObservedRunningTime="2026-09-06 10:46:52.538739562 +0000 UTC m=+1024.988520330"
Sep 06 10:46:52.808827 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:52.808743    2557 patch_prober.go:28] interesting pod/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 06 10:46:52.808827 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:52.808798    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5" podUID="475cab92-806a-4de2-bc9b-2b33485eba74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 06 10:46:53.147412 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:53.147375    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:53.152655 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:53.152629    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:53.525040 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:53.524953    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:53.526054 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:53.526030    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:46:53.809532 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:53.809418    2557 patch_prober.go:28] interesting pod/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 06 10:46:53.809532 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:53.809481    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5" podUID="475cab92-806a-4de2-bc9b-2b33485eba74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 06 10:46:54.812660 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:54.812629    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:54.813030 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:54.813009    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:46:54.813748 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:46:54.813731    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:47:02.318675 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:02.318641    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:47:13.647343 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:13.647309    2557 generic.go:356] "Generic (PLEG): container finished" podID="3b061474-63c2-4ca7-b309-189cb3bad4d7" containerID="dc8a68620f3e1b12c5e3fa266418e280d069bb9de89bf88c93550f5f4d81da2c" exitCode=0
Sep 06 10:47:13.647760 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:13.647354    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff" event={"ID":"3b061474-63c2-4ca7-b309-189cb3bad4d7","Type":"ContainerDied","Data":"dc8a68620f3e1b12c5e3fa266418e280d069bb9de89bf88c93550f5f4d81da2c"}
Sep 06 10:47:13.647760 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:13.647713    2557 scope.go:117] "RemoveContainer" containerID="dc8a68620f3e1b12c5e3fa266418e280d069bb9de89bf88c93550f5f4d81da2c"
Sep 06 10:47:13.648730 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:13.648702    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:47:13.648791 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:13.648769    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:47:13.648833 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:13.648802    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:47:14.661272 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:14.661238    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff" event={"ID":"3b061474-63c2-4ca7-b309-189cb3bad4d7","Type":"ContainerStarted","Data":"21da10318eb1d0f27b53a1f1e487fd1844d2e2c94dc0daad4e4b27ad28358e48"}
Sep 06 10:47:15.147532 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:15.147482    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:47:15.147817 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:15.147792    2557 patch_prober.go:28] interesting pod/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 06 10:47:15.147869 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:15.147845    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff" podUID="3b061474-63c2-4ca7-b309-189cb3bad4d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 06 10:47:16.147575 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:16.147535    2557 patch_prober.go:28] interesting pod/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 06 10:47:16.147954 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:16.147610    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff" podUID="3b061474-63c2-4ca7-b309-189cb3bad4d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 06 10:47:17.147874 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:17.147834    2557 patch_prober.go:28] interesting pod/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 06 10:47:17.148252 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:17.147922    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff" podUID="3b061474-63c2-4ca7-b309-189cb3bad4d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 06 10:47:18.147535 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:18.147481    2557 patch_prober.go:28] interesting pod/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 06 10:47:18.147709 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:18.147556    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff" podUID="3b061474-63c2-4ca7-b309-189cb3bad4d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 06 10:47:18.326679 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:18.326649    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:47:19.148173 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:19.148132    2557 patch_prober.go:28] interesting pod/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 06 10:47:19.148376 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:19.148200    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff" podUID="3b061474-63c2-4ca7-b309-189cb3bad4d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 06 10:47:19.994917 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:19.994883    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7817-140606201255232"]
Sep 06 10:47:19.997395 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:19.997364    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7817-140606201255232"
Sep 06 10:47:19.999631 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:19.999612    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lwnht\""
Sep 06 10:47:20.006430 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:20.005929    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7817-140606201255232"]
Sep 06 10:47:20.077941 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:20.077909    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfmrd\" (UniqueName: \"kubernetes.io/projected/847f71d7-12ae-42d2-b543-862413f0754a-kube-api-access-gfmrd\") pod \"test-curl-7817-140606201255232\" (UID: \"847f71d7-12ae-42d2-b543-862413f0754a\") " pod="opendatahub/test-curl-7817-140606201255232"
Sep 06 10:47:20.152618 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:20.152589    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:47:20.152911 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:20.152887    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:47:20.153443 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:20.153422    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:47:20.178683 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:20.178652    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfmrd\" (UniqueName: \"kubernetes.io/projected/847f71d7-12ae-42d2-b543-862413f0754a-kube-api-access-gfmrd\") pod \"test-curl-7817-140606201255232\" (UID: \"847f71d7-12ae-42d2-b543-862413f0754a\") " pod="opendatahub/test-curl-7817-140606201255232"
Sep 06 10:47:20.189325 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:20.189300    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfmrd\" (UniqueName: \"kubernetes.io/projected/847f71d7-12ae-42d2-b543-862413f0754a-kube-api-access-gfmrd\") pod \"test-curl-7817-140606201255232\" (UID: \"847f71d7-12ae-42d2-b543-862413f0754a\") " pod="opendatahub/test-curl-7817-140606201255232"
Sep 06 10:47:20.310117 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:20.310036    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7817-140606201255232"
Sep 06 10:47:20.577093 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:20.577059    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7817-140606201255232"]
Sep 06 10:47:20.578529 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:47:20.578501    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod847f71d7_12ae_42d2_b543_862413f0754a.slice/crio-da8bf4eef408b204905e0591a442631a9099110d9e81254d64025e45f2a8004e WatchSource:0}: Error finding container da8bf4eef408b204905e0591a442631a9099110d9e81254d64025e45f2a8004e: Status 404 returned error can't find the container with id da8bf4eef408b204905e0591a442631a9099110d9e81254d64025e45f2a8004e
Sep 06 10:47:20.580201 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:20.580181    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 10:47:20.705893 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:20.705853    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7817-140606201255232" event={"ID":"847f71d7-12ae-42d2-b543-862413f0754a","Type":"ContainerStarted","Data":"da8bf4eef408b204905e0591a442631a9099110d9e81254d64025e45f2a8004e"}
Sep 06 10:47:21.713274 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:21.713238    2557 generic.go:356] "Generic (PLEG): container finished" podID="847f71d7-12ae-42d2-b543-862413f0754a" containerID="7b5075514643807929caf7ec8a55b0b8d66cd666366774c1c6cd50e4ea1e20f3" exitCode=0
Sep 06 10:47:21.713692 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:21.713273    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7817-140606201255232" event={"ID":"847f71d7-12ae-42d2-b543-862413f0754a","Type":"ContainerDied","Data":"7b5075514643807929caf7ec8a55b0b8d66cd666366774c1c6cd50e4ea1e20f3"}
Sep 06 10:47:22.856926 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:22.856903    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7817-140606201255232"
Sep 06 10:47:22.888035 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:22.888010    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7817-140606201255232_847f71d7-12ae-42d2-b543-862413f0754a/test-curl-7817-140606201255232/0.log"
Sep 06 10:47:22.905514 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:22.905478    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfmrd\" (UniqueName: \"kubernetes.io/projected/847f71d7-12ae-42d2-b543-862413f0754a-kube-api-access-gfmrd\") pod \"847f71d7-12ae-42d2-b543-862413f0754a\" (UID: \"847f71d7-12ae-42d2-b543-862413f0754a\") "
Sep 06 10:47:22.907720 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:22.907693    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/847f71d7-12ae-42d2-b543-862413f0754a-kube-api-access-gfmrd" (OuterVolumeSpecName: "kube-api-access-gfmrd") pod "847f71d7-12ae-42d2-b543-862413f0754a" (UID: "847f71d7-12ae-42d2-b543-862413f0754a"). InnerVolumeSpecName "kube-api-access-gfmrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:47:22.908667 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:22.908645    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7817-140606201255232"]
Sep 06 10:47:22.911746 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:22.911726    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7817-140606201255232"]
Sep 06 10:47:23.006559 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:23.006459    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfmrd\" (UniqueName: \"kubernetes.io/projected/847f71d7-12ae-42d2-b543-862413f0754a-kube-api-access-gfmrd\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:47:23.026958 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:23.026925    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7817-140606201255232"]
Sep 06 10:47:23.027316 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:23.027303    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="847f71d7-12ae-42d2-b543-862413f0754a" containerName="test-curl-7817-140606201255232"
Sep 06 10:47:23.027355 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:23.027317    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="847f71d7-12ae-42d2-b543-862413f0754a" containerName="test-curl-7817-140606201255232"
Sep 06 10:47:23.027409 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:23.027400    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="847f71d7-12ae-42d2-b543-862413f0754a" containerName="test-curl-7817-140606201255232"
Sep 06 10:47:23.030125 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:23.030108    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7817-140606201255232"
Sep 06 10:47:23.033630 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:23.033605    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7817-140606201255232"]
Sep 06 10:47:23.107356 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:23.107323    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnh76\" (UniqueName: \"kubernetes.io/projected/ac9ffcb8-6ee6-4c2a-b85f-71e86550124f-kube-api-access-vnh76\") pod \"test-curl-7817-140606201255232\" (UID: \"ac9ffcb8-6ee6-4c2a-b85f-71e86550124f\") " pod="opendatahub/test-curl-7817-140606201255232"
Sep 06 10:47:23.208638 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:23.208600    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnh76\" (UniqueName: \"kubernetes.io/projected/ac9ffcb8-6ee6-4c2a-b85f-71e86550124f-kube-api-access-vnh76\") pod \"test-curl-7817-140606201255232\" (UID: \"ac9ffcb8-6ee6-4c2a-b85f-71e86550124f\") " pod="opendatahub/test-curl-7817-140606201255232"
Sep 06 10:47:23.216199 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:23.216171    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnh76\" (UniqueName: \"kubernetes.io/projected/ac9ffcb8-6ee6-4c2a-b85f-71e86550124f-kube-api-access-vnh76\") pod \"test-curl-7817-140606201255232\" (UID: \"ac9ffcb8-6ee6-4c2a-b85f-71e86550124f\") " pod="opendatahub/test-curl-7817-140606201255232"
Sep 06 10:47:23.340672 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:23.340646    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7817-140606201255232"
Sep 06 10:47:23.611843 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:23.611819    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7817-140606201255232"]
Sep 06 10:47:23.613730 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:47:23.613704    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac9ffcb8_6ee6_4c2a_b85f_71e86550124f.slice/crio-d20771d54a1a0cddf94b3f186a352bec9596562155cf9f724e886068d9079cf8 WatchSource:0}: Error finding container d20771d54a1a0cddf94b3f186a352bec9596562155cf9f724e886068d9079cf8: Status 404 returned error can't find the container with id d20771d54a1a0cddf94b3f186a352bec9596562155cf9f724e886068d9079cf8
Sep 06 10:47:23.725663 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:23.725626    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7817-140606201255232" event={"ID":"ac9ffcb8-6ee6-4c2a-b85f-71e86550124f","Type":"ContainerStarted","Data":"d20771d54a1a0cddf94b3f186a352bec9596562155cf9f724e886068d9079cf8"}
Sep 06 10:47:23.726848 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:23.726823    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7817-140606201255232"
Sep 06 10:47:23.726980 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:23.726848    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da8bf4eef408b204905e0591a442631a9099110d9e81254d64025e45f2a8004e"
Sep 06 10:47:23.737896 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:23.737855    2557 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7817-140606201255232" oldPodUID="847f71d7-12ae-42d2-b543-862413f0754a" podUID="ac9ffcb8-6ee6-4c2a-b85f-71e86550124f"
Sep 06 10:47:24.037071 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:24.036991    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="847f71d7-12ae-42d2-b543-862413f0754a" path="/var/lib/kubelet/pods/847f71d7-12ae-42d2-b543-862413f0754a/volumes"
Sep 06 10:47:24.734250 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:24.734154    2557 generic.go:356] "Generic (PLEG): container finished" podID="ac9ffcb8-6ee6-4c2a-b85f-71e86550124f" containerID="537f81696552519439d427cf561cdf3961f5bda9081080fc7a7990831755223b" exitCode=0
Sep 06 10:47:24.734250 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:24.734201    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7817-140606201255232" event={"ID":"ac9ffcb8-6ee6-4c2a-b85f-71e86550124f","Type":"ContainerDied","Data":"537f81696552519439d427cf561cdf3961f5bda9081080fc7a7990831755223b"}
Sep 06 10:47:25.875467 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:25.875441    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7817-140606201255232"
Sep 06 10:47:25.893360 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:25.893317    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7817-140606201255232_ac9ffcb8-6ee6-4c2a-b85f-71e86550124f/test-curl-7817-140606201255232/0.log"
Sep 06 10:47:25.914602 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:25.914568    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7817-140606201255232"]
Sep 06 10:47:25.919459 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:25.919434    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7817-140606201255232"]
Sep 06 10:47:25.935670 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:25.935648    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnh76\" (UniqueName: \"kubernetes.io/projected/ac9ffcb8-6ee6-4c2a-b85f-71e86550124f-kube-api-access-vnh76\") pod \"ac9ffcb8-6ee6-4c2a-b85f-71e86550124f\" (UID: \"ac9ffcb8-6ee6-4c2a-b85f-71e86550124f\") "
Sep 06 10:47:25.937769 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:25.937747    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac9ffcb8-6ee6-4c2a-b85f-71e86550124f-kube-api-access-vnh76" (OuterVolumeSpecName: "kube-api-access-vnh76") pod "ac9ffcb8-6ee6-4c2a-b85f-71e86550124f" (UID: "ac9ffcb8-6ee6-4c2a-b85f-71e86550124f"). InnerVolumeSpecName "kube-api-access-vnh76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:47:26.036856 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:26.036775    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnh76\" (UniqueName: \"kubernetes.io/projected/ac9ffcb8-6ee6-4c2a-b85f-71e86550124f-kube-api-access-vnh76\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:47:26.037829 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:26.037803    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac9ffcb8-6ee6-4c2a-b85f-71e86550124f" path="/var/lib/kubelet/pods/ac9ffcb8-6ee6-4c2a-b85f-71e86550124f/volumes"
Sep 06 10:47:26.038155 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:26.038141    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7817-140606201255808"]
Sep 06 10:47:26.038530 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:26.038513    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac9ffcb8-6ee6-4c2a-b85f-71e86550124f" containerName="test-curl-7817-140606201255232"
Sep 06 10:47:26.038530 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:26.038527    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac9ffcb8-6ee6-4c2a-b85f-71e86550124f" containerName="test-curl-7817-140606201255232"
Sep 06 10:47:26.038696 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:26.038606    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac9ffcb8-6ee6-4c2a-b85f-71e86550124f" containerName="test-curl-7817-140606201255232"
Sep 06 10:47:26.041725 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:26.041706    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7817-140606201255808"
Sep 06 10:47:26.045382 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:26.045359    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7817-140606201255808"]
Sep 06 10:47:26.137541 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:26.137481    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvpz2\" (UniqueName: \"kubernetes.io/projected/c6d38466-bebb-41d2-ac4f-e3914fc5493a-kube-api-access-fvpz2\") pod \"test-curl-7817-140606201255808\" (UID: \"c6d38466-bebb-41d2-ac4f-e3914fc5493a\") " pod="opendatahub/test-curl-7817-140606201255808"
Sep 06 10:47:26.239075 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:26.239037    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvpz2\" (UniqueName: \"kubernetes.io/projected/c6d38466-bebb-41d2-ac4f-e3914fc5493a-kube-api-access-fvpz2\") pod \"test-curl-7817-140606201255808\" (UID: \"c6d38466-bebb-41d2-ac4f-e3914fc5493a\") " pod="opendatahub/test-curl-7817-140606201255808"
Sep 06 10:47:26.247097 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:26.247075    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvpz2\" (UniqueName: \"kubernetes.io/projected/c6d38466-bebb-41d2-ac4f-e3914fc5493a-kube-api-access-fvpz2\") pod \"test-curl-7817-140606201255808\" (UID: \"c6d38466-bebb-41d2-ac4f-e3914fc5493a\") " pod="opendatahub/test-curl-7817-140606201255808"
Sep 06 10:47:26.355335 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:26.355301    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7817-140606201255808"
Sep 06 10:47:26.623670 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:26.623644    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7817-140606201255808"]
Sep 06 10:47:26.625903 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:47:26.625870    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6d38466_bebb_41d2_ac4f_e3914fc5493a.slice/crio-400137d58c343993546e2528136cee8e71af6396727d0f89641290a0a0686576 WatchSource:0}: Error finding container 400137d58c343993546e2528136cee8e71af6396727d0f89641290a0a0686576: Status 404 returned error can't find the container with id 400137d58c343993546e2528136cee8e71af6396727d0f89641290a0a0686576
Sep 06 10:47:26.746327 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:26.746295    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7817-140606201255232"
Sep 06 10:47:26.746544 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:26.746336    2557 scope.go:117] "RemoveContainer" containerID="537f81696552519439d427cf561cdf3961f5bda9081080fc7a7990831755223b"
Sep 06 10:47:26.747757 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:26.747707    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7817-140606201255808" event={"ID":"c6d38466-bebb-41d2-ac4f-e3914fc5493a","Type":"ContainerStarted","Data":"400137d58c343993546e2528136cee8e71af6396727d0f89641290a0a0686576"}
Sep 06 10:47:27.755761 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:27.755722    2557 generic.go:356] "Generic (PLEG): container finished" podID="c6d38466-bebb-41d2-ac4f-e3914fc5493a" containerID="da0037da7dee09fdb35b66bc4fec346de13c9a850735ea1baa6b61583ede2ac0" exitCode=0
Sep 06 10:47:27.755761 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:27.755762    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7817-140606201255808" event={"ID":"c6d38466-bebb-41d2-ac4f-e3914fc5493a","Type":"ContainerDied","Data":"da0037da7dee09fdb35b66bc4fec346de13c9a850735ea1baa6b61583ede2ac0"}
Sep 06 10:47:28.893650 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:28.893627    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7817-140606201255808"
Sep 06 10:47:28.912658 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:28.912627    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7817-140606201255808_c6d38466-bebb-41d2-ac4f-e3914fc5493a/test-curl-7817-140606201255808/0.log"
Sep 06 10:47:28.933644 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:28.933610    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7817-140606201255808"]
Sep 06 10:47:28.935571 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:28.935544    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7817-140606201255808"]
Sep 06 10:47:28.962099 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:28.962076    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvpz2\" (UniqueName: \"kubernetes.io/projected/c6d38466-bebb-41d2-ac4f-e3914fc5493a-kube-api-access-fvpz2\") pod \"c6d38466-bebb-41d2-ac4f-e3914fc5493a\" (UID: \"c6d38466-bebb-41d2-ac4f-e3914fc5493a\") "
Sep 06 10:47:28.964140 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:28.964111    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6d38466-bebb-41d2-ac4f-e3914fc5493a-kube-api-access-fvpz2" (OuterVolumeSpecName: "kube-api-access-fvpz2") pod "c6d38466-bebb-41d2-ac4f-e3914fc5493a" (UID: "c6d38466-bebb-41d2-ac4f-e3914fc5493a"). InnerVolumeSpecName "kube-api-access-fvpz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:47:29.060100 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:29.060022    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7817-140606201255808"]
Sep 06 10:47:29.060428 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:29.060416    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6d38466-bebb-41d2-ac4f-e3914fc5493a" containerName="test-curl-7817-140606201255808"
Sep 06 10:47:29.060476 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:29.060431    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6d38466-bebb-41d2-ac4f-e3914fc5493a" containerName="test-curl-7817-140606201255808"
Sep 06 10:47:29.060550 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:29.060513    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6d38466-bebb-41d2-ac4f-e3914fc5493a" containerName="test-curl-7817-140606201255808"
Sep 06 10:47:29.063323 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:29.063292    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvpz2\" (UniqueName: \"kubernetes.io/projected/c6d38466-bebb-41d2-ac4f-e3914fc5493a-kube-api-access-fvpz2\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:47:29.063470 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:29.063447    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7817-140606201255808"
Sep 06 10:47:29.068899 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:29.068875    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7817-140606201255808"]
Sep 06 10:47:29.164808 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:29.164771    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p46fc\" (UniqueName: \"kubernetes.io/projected/65f38e5e-5dea-468d-9c69-97c35010efc7-kube-api-access-p46fc\") pod \"test-curl-7817-140606201255808\" (UID: \"65f38e5e-5dea-468d-9c69-97c35010efc7\") " pod="opendatahub/test-curl-7817-140606201255808"
Sep 06 10:47:29.265545 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:29.265483    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p46fc\" (UniqueName: \"kubernetes.io/projected/65f38e5e-5dea-468d-9c69-97c35010efc7-kube-api-access-p46fc\") pod \"test-curl-7817-140606201255808\" (UID: \"65f38e5e-5dea-468d-9c69-97c35010efc7\") " pod="opendatahub/test-curl-7817-140606201255808"
Sep 06 10:47:29.273516 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:29.273466    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p46fc\" (UniqueName: \"kubernetes.io/projected/65f38e5e-5dea-468d-9c69-97c35010efc7-kube-api-access-p46fc\") pod \"test-curl-7817-140606201255808\" (UID: \"65f38e5e-5dea-468d-9c69-97c35010efc7\") " pod="opendatahub/test-curl-7817-140606201255808"
Sep 06 10:47:29.375624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:29.375586    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7817-140606201255808"
Sep 06 10:47:29.646412 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:29.646387    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7817-140606201255808"]
Sep 06 10:47:29.648583 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:47:29.648550    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65f38e5e_5dea_468d_9c69_97c35010efc7.slice/crio-4aefac16472d3ef65e0bd82f484e99abacd91233620ac857e6af1775b8f2758a WatchSource:0}: Error finding container 4aefac16472d3ef65e0bd82f484e99abacd91233620ac857e6af1775b8f2758a: Status 404 returned error can't find the container with id 4aefac16472d3ef65e0bd82f484e99abacd91233620ac857e6af1775b8f2758a
Sep 06 10:47:29.768342 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:29.768310    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7817-140606201255808" event={"ID":"65f38e5e-5dea-468d-9c69-97c35010efc7","Type":"ContainerStarted","Data":"4aefac16472d3ef65e0bd82f484e99abacd91233620ac857e6af1775b8f2758a"}
Sep 06 10:47:29.769742 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:29.769714    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7817-140606201255808"
Sep 06 10:47:29.769886 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:29.769749    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="400137d58c343993546e2528136cee8e71af6396727d0f89641290a0a0686576"
Sep 06 10:47:29.781363 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:29.781336    2557 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7817-140606201255808" oldPodUID="c6d38466-bebb-41d2-ac4f-e3914fc5493a" podUID="65f38e5e-5dea-468d-9c69-97c35010efc7"
Sep 06 10:47:30.039188 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:30.039101    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6d38466-bebb-41d2-ac4f-e3914fc5493a" path="/var/lib/kubelet/pods/c6d38466-bebb-41d2-ac4f-e3914fc5493a/volumes"
Sep 06 10:47:30.777184 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:30.777102    2557 generic.go:356] "Generic (PLEG): container finished" podID="65f38e5e-5dea-468d-9c69-97c35010efc7" containerID="b8bf17a59c36eff0d5c0bf1dfa8e220b0552a5c7d285bc7d31dca4c034b3e00c" exitCode=0
Sep 06 10:47:30.777331 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:30.777186    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7817-140606201255808" event={"ID":"65f38e5e-5dea-468d-9c69-97c35010efc7","Type":"ContainerDied","Data":"b8bf17a59c36eff0d5c0bf1dfa8e220b0552a5c7d285bc7d31dca4c034b3e00c"}
Sep 06 10:47:31.925178 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:31.925154    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7817-140606201255808"
Sep 06 10:47:31.942888 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:31.942866    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7817-140606201255808_65f38e5e-5dea-468d-9c69-97c35010efc7/test-curl-7817-140606201255808/0.log"
Sep 06 10:47:31.966572 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:31.966535    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7817-140606201255808"]
Sep 06 10:47:31.968314 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:31.968291    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7817-140606201255808"]
Sep 06 10:47:31.989995 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:31.989970    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p46fc\" (UniqueName: \"kubernetes.io/projected/65f38e5e-5dea-468d-9c69-97c35010efc7-kube-api-access-p46fc\") pod \"65f38e5e-5dea-468d-9c69-97c35010efc7\" (UID: \"65f38e5e-5dea-468d-9c69-97c35010efc7\") "
Sep 06 10:47:31.992054 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:31.992029    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65f38e5e-5dea-468d-9c69-97c35010efc7-kube-api-access-p46fc" (OuterVolumeSpecName: "kube-api-access-p46fc") pod "65f38e5e-5dea-468d-9c69-97c35010efc7" (UID: "65f38e5e-5dea-468d-9c69-97c35010efc7"). InnerVolumeSpecName "kube-api-access-p46fc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:47:32.037348 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:32.037277    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65f38e5e-5dea-468d-9c69-97c35010efc7" path="/var/lib/kubelet/pods/65f38e5e-5dea-468d-9c69-97c35010efc7/volumes"
Sep 06 10:47:32.091938 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:32.091891    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p46fc\" (UniqueName: \"kubernetes.io/projected/65f38e5e-5dea-468d-9c69-97c35010efc7-kube-api-access-p46fc\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:47:32.791763 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:32.791733    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7817-140606201255808"
Sep 06 10:47:32.791945 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:32.791782    2557 scope.go:117] "RemoveContainer" containerID="b8bf17a59c36eff0d5c0bf1dfa8e220b0552a5c7d285bc7d31dca4c034b3e00c"
Sep 06 10:47:32.839064 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:32.839034    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"]
Sep 06 10:47:32.839473 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:32.839457    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65f38e5e-5dea-468d-9c69-97c35010efc7" containerName="test-curl-7817-140606201255808"
Sep 06 10:47:32.839473 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:32.839473    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="65f38e5e-5dea-468d-9c69-97c35010efc7" containerName="test-curl-7817-140606201255808"
Sep 06 10:47:32.839660 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:32.839567    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="65f38e5e-5dea-468d-9c69-97c35010efc7" containerName="test-curl-7817-140606201255808"
Sep 06 10:47:32.844179 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:32.844156    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:32.846427 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:32.846404    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-aec3c2bf-openshift-default-dockercfg-c577d\""
Sep 06 10:47:32.852104 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:32.852079    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"]
Sep 06 10:47:33.000890 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.000851    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e635cc2-998d-4b39-ba2a-db7501658996-istio-podinfo\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.000890 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.000894    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-credential-socket\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.001401 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.000921    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-workload-certs\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.001401 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.000953    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e635cc2-998d-4b39-ba2a-db7501658996-istiod-ca-cert\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.001401 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.001000    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnlks\" (UniqueName: \"kubernetes.io/projected/0e635cc2-998d-4b39-ba2a-db7501658996-kube-api-access-xnlks\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.001401 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.001033    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-workload-socket\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.001401 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.001071    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-istio-envoy\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.001401 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.001158    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-istio-data\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.001401 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.001200    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e635cc2-998d-4b39-ba2a-db7501658996-istio-token\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.101706 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.101670    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e635cc2-998d-4b39-ba2a-db7501658996-istio-token\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.101900 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.101727    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e635cc2-998d-4b39-ba2a-db7501658996-istio-podinfo\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.101900 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.101763    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-credential-socket\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.101900 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.101791    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-workload-certs\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.101900 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.101815    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e635cc2-998d-4b39-ba2a-db7501658996-istiod-ca-cert\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.101900 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.101845    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnlks\" (UniqueName: \"kubernetes.io/projected/0e635cc2-998d-4b39-ba2a-db7501658996-kube-api-access-xnlks\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.101900 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.101873    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-workload-socket\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.102207 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.101917    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-istio-envoy\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.102207 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.102005    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-istio-data\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.102316 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.102202    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-workload-certs\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.102316 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.102214    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-credential-socket\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.102425 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.102336    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-workload-socket\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.102425 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.102412    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-istio-data\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.102912 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.102888    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e635cc2-998d-4b39-ba2a-db7501658996-istiod-ca-cert\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.104172 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.104147    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-istio-envoy\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.104422 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.104402    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e635cc2-998d-4b39-ba2a-db7501658996-istio-podinfo\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.112247 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.112219    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e635cc2-998d-4b39-ba2a-db7501658996-istio-token\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.112531 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.112511    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnlks\" (UniqueName: \"kubernetes.io/projected/0e635cc2-998d-4b39-ba2a-db7501658996-kube-api-access-xnlks\") pod \"e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") " pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.157529 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.157485    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:33.431061 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.431036    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"]
Sep 06 10:47:33.432459 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:47:33.432429    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e635cc2_998d_4b39_ba2a_db7501658996.slice/crio-0b692b2babfa5a68c5fd33d26662e23fcd38da45e38d0c6fd7ddb72845bd9483 WatchSource:0}: Error finding container 0b692b2babfa5a68c5fd33d26662e23fcd38da45e38d0c6fd7ddb72845bd9483: Status 404 returned error can't find the container with id 0b692b2babfa5a68c5fd33d26662e23fcd38da45e38d0c6fd7ddb72845bd9483
Sep 06 10:47:33.435052 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.435006    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:47:33.435125 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.435107    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:47:33.435170 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.435155    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:47:33.798739 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.798638    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q" event={"ID":"0e635cc2-998d-4b39-ba2a-db7501658996","Type":"ContainerStarted","Data":"327be180c07fb48a892651421d913d90a1a89cd6af64b4572c87d68c218d30bd"}
Sep 06 10:47:33.798739 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.798737    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q" event={"ID":"0e635cc2-998d-4b39-ba2a-db7501658996","Type":"ContainerStarted","Data":"0b692b2babfa5a68c5fd33d26662e23fcd38da45e38d0c6fd7ddb72845bd9483"}
Sep 06 10:47:33.817943 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:33.817895    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q" podStartSLOduration=1.817881136 podStartE2EDuration="1.817881136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:47:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:47:33.816526432 +0000 UTC m=+1066.266307219" watchObservedRunningTime="2026-09-06 10:47:33.817881136 +0000 UTC m=+1066.267661903"
Sep 06 10:47:34.157664 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:34.157629    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:34.162965 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:34.162939    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:34.806397 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:34.806370    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:34.807229 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:34.807209    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:36.915979 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:36.915946    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"]
Sep 06 10:47:36.919764 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:36.919737    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"
Sep 06 10:47:36.922008 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:36.921980    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-aec3c2bf-serving-cert\""
Sep 06 10:47:36.922341 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:36.922318    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 06 10:47:36.923215 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:36.923190    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 10:47:36.923328 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:36.923190    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 06 10:47:36.923328 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:36.923194    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q972h\""
Sep 06 10:47:36.928056 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:36.928032    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"]
Sep 06 10:47:37.043297 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:37.043263    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-maas-api-tls\") pod \"maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr\" (UID: \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"
Sep 06 10:47:37.043458 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:37.043385    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-metrics-certs\") pod \"maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr\" (UID: \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"
Sep 06 10:47:37.043458 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:37.043425    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-openshift-service-ca\") pod \"maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr\" (UID: \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"
Sep 06 10:47:37.043458 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:37.043451    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xm78\" (UniqueName: \"kubernetes.io/projected/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-kube-api-access-7xm78\") pod \"maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr\" (UID: \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"
Sep 06 10:47:37.144427 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:37.144392    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-maas-api-tls\") pod \"maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr\" (UID: \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"
Sep 06 10:47:37.144612 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:37.144468    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-metrics-certs\") pod \"maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr\" (UID: \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"
Sep 06 10:47:37.144612 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:37.144524    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-openshift-service-ca\") pod \"maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr\" (UID: \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"
Sep 06 10:47:37.144612 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:37.144556    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xm78\" (UniqueName: \"kubernetes.io/projected/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-kube-api-access-7xm78\") pod \"maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr\" (UID: \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"
Sep 06 10:47:37.144612 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:47:37.144561    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-serving-cert: secret "maas-api-e2e-mt-aec3c2bf-serving-cert" not found
Sep 06 10:47:37.144807 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:47:37.144628    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-maas-api-tls podName:3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac nodeName:}" failed. No retries permitted until 2026-09-06 10:47:37.6446116 +0000 UTC m=+1070.094392369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-maas-api-tls") pod "maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr" (UID: "3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac") : secret "maas-api-e2e-mt-aec3c2bf-serving-cert" not found
Sep 06 10:47:37.145274 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:37.145250    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-openshift-service-ca\") pod \"maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr\" (UID: \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"
Sep 06 10:47:37.146937 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:37.146917    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-metrics-certs\") pod \"maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr\" (UID: \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"
Sep 06 10:47:37.152621 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:37.152597    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xm78\" (UniqueName: \"kubernetes.io/projected/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-kube-api-access-7xm78\") pod \"maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr\" (UID: \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"
Sep 06 10:47:37.650689 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:37.650636    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-maas-api-tls\") pod \"maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr\" (UID: \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"
Sep 06 10:47:37.653639 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:37.653606    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-maas-api-tls\") pod \"maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr\" (UID: \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"
Sep 06 10:47:37.835083 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:37.835047    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"
Sep 06 10:47:38.113362 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:38.113332    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"]
Sep 06 10:47:38.114463 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:47:38.114442    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ca59ad2_7f5f_44b6_9084_f2bc6c28e6ac.slice/crio-940bd353b60a1c441e0e88df9a4fd439a335b4e1d359c50a5b7d36eac3658c03 WatchSource:0}: Error finding container 940bd353b60a1c441e0e88df9a4fd439a335b4e1d359c50a5b7d36eac3658c03: Status 404 returned error can't find the container with id 940bd353b60a1c441e0e88df9a4fd439a335b4e1d359c50a5b7d36eac3658c03
Sep 06 10:47:38.835556 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:38.835517    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr" event={"ID":"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac","Type":"ContainerStarted","Data":"940bd353b60a1c441e0e88df9a4fd439a335b4e1d359c50a5b7d36eac3658c03"}
Sep 06 10:47:39.843512 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:39.843464    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr" event={"ID":"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac","Type":"ContainerStarted","Data":"aa5cbcc660fe2d2a4a7ba8e40678441ac33c7bf71adeec0342fa847a45e146d5"}
Sep 06 10:47:39.843872 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:39.843557    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"
Sep 06 10:47:39.859065 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:39.859011    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr" podStartSLOduration=2.304776398 podStartE2EDuration="3.858995118s" totalImagesPullingTime="1.55421872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:47:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:47:39.85805585 +0000 UTC m=+1072.307836620" watchObservedRunningTime="2026-09-06 10:47:39.858995118 +0000 UTC m=+1072.308775886"
Sep 06 10:47:44.921287 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:44.921202    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:47:45.855029 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:45.854998    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"
Sep 06 10:47:48.184020 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:48.183981    2557 scope.go:117] "RemoveContainer" containerID="d4b19881a0ac8af76623765a5aa9d7b71988652a1342308298e973435a52998a"
Sep 06 10:47:55.946120 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:55.946077    2557 generic.go:356] "Generic (PLEG): container finished" podID="0e635cc2-998d-4b39-ba2a-db7501658996" containerID="327be180c07fb48a892651421d913d90a1a89cd6af64b4572c87d68c218d30bd" exitCode=0
Sep 06 10:47:55.946574 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:55.946143    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q" event={"ID":"0e635cc2-998d-4b39-ba2a-db7501658996","Type":"ContainerDied","Data":"327be180c07fb48a892651421d913d90a1a89cd6af64b4572c87d68c218d30bd"}
Sep 06 10:47:55.946624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:55.946602    2557 scope.go:117] "RemoveContainer" containerID="327be180c07fb48a892651421d913d90a1a89cd6af64b4572c87d68c218d30bd"
Sep 06 10:47:55.947746 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:55.947707    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:47:55.947806 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:55.947790    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:47:55.947847 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:55.947822    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:47:56.955223 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:56.955183    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q" event={"ID":"0e635cc2-998d-4b39-ba2a-db7501658996","Type":"ContainerStarted","Data":"b08efdb2174e76dba65842c38c37dc2518b033d86c4ee37edfa3a2d342e19794"}
Sep 06 10:47:57.158384 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:57.158343    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:47:57.158650 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:57.158626    2557 patch_prober.go:28] interesting pod/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 06 10:47:57.158701 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:57.158678    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q" podUID="0e635cc2-998d-4b39-ba2a-db7501658996" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 06 10:47:58.158003 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:58.157964    2557 patch_prober.go:28] interesting pod/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 06 10:47:58.158419 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:58.158025    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q" podUID="0e635cc2-998d-4b39-ba2a-db7501658996" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 06 10:47:59.158797 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:59.158758    2557 patch_prober.go:28] interesting pod/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 06 10:47:59.159206 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:47:59.158815    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q" podUID="0e635cc2-998d-4b39-ba2a-db7501658996" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 06 10:48:00.158415 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:48:00.158374    2557 patch_prober.go:28] interesting pod/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 06 10:48:00.158631 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:48:00.158452    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q" podUID="0e635cc2-998d-4b39-ba2a-db7501658996" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 06 10:48:01.158335 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:48:01.158300    2557 patch_prober.go:28] interesting pod/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 06 10:48:01.158836 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:48:01.158375    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q" podUID="0e635cc2-998d-4b39-ba2a-db7501658996" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 06 10:48:02.163830 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:48:02.163798    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:48:02.164296 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:48:02.164164    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:48:02.165023 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:48:02.165004    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:48:20.316797 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:48:20.316766    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:48:34.014039 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:48:34.014001    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:48:47.614503 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:48:47.614449    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:48:48.220323 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:48:48.220291    2557 scope.go:117] "RemoveContainer" containerID="400b3191a7d7faf0c019ec529640a94d6b0eb8924e16f5880a2326735e144f9a"
Sep 06 10:49:02.614445 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:02.614408    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:49:11.140187 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.140099    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"]
Sep 06 10:49:11.140769 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.140333    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr" podUID="3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac" containerName="maas-api" containerID="cri-o://aa5cbcc660fe2d2a4a7ba8e40678441ac33c7bf71adeec0342fa847a45e146d5" gracePeriod=30
Sep 06 10:49:11.385459 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.385433    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"
Sep 06 10:49:11.414243 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.414172    2557 generic.go:356] "Generic (PLEG): container finished" podID="3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac" containerID="aa5cbcc660fe2d2a4a7ba8e40678441ac33c7bf71adeec0342fa847a45e146d5" exitCode=0
Sep 06 10:49:11.414243 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.414219    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"
Sep 06 10:49:11.414434 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.414244    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr" event={"ID":"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac","Type":"ContainerDied","Data":"aa5cbcc660fe2d2a4a7ba8e40678441ac33c7bf71adeec0342fa847a45e146d5"}
Sep 06 10:49:11.414434 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.414293    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr" event={"ID":"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac","Type":"ContainerDied","Data":"940bd353b60a1c441e0e88df9a4fd439a335b4e1d359c50a5b7d36eac3658c03"}
Sep 06 10:49:11.414434 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.414316    2557 scope.go:117] "RemoveContainer" containerID="aa5cbcc660fe2d2a4a7ba8e40678441ac33c7bf71adeec0342fa847a45e146d5"
Sep 06 10:49:11.424863 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.424837    2557 scope.go:117] "RemoveContainer" containerID="aa5cbcc660fe2d2a4a7ba8e40678441ac33c7bf71adeec0342fa847a45e146d5"
Sep 06 10:49:11.425188 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:49:11.425154    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa5cbcc660fe2d2a4a7ba8e40678441ac33c7bf71adeec0342fa847a45e146d5\": container with ID starting with aa5cbcc660fe2d2a4a7ba8e40678441ac33c7bf71adeec0342fa847a45e146d5 not found: ID does not exist" containerID="aa5cbcc660fe2d2a4a7ba8e40678441ac33c7bf71adeec0342fa847a45e146d5"
Sep 06 10:49:11.425261 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.425203    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa5cbcc660fe2d2a4a7ba8e40678441ac33c7bf71adeec0342fa847a45e146d5"} err="failed to get container status \"aa5cbcc660fe2d2a4a7ba8e40678441ac33c7bf71adeec0342fa847a45e146d5\": rpc error: code = NotFound desc = could not find container \"aa5cbcc660fe2d2a4a7ba8e40678441ac33c7bf71adeec0342fa847a45e146d5\": container with ID starting with aa5cbcc660fe2d2a4a7ba8e40678441ac33c7bf71adeec0342fa847a45e146d5 not found: ID does not exist"
Sep 06 10:49:11.560929 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.560902    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-maas-api-tls\") pod \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\" (UID: \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\") "
Sep 06 10:49:11.561111 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.560937    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-metrics-certs\") pod \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\" (UID: \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\") "
Sep 06 10:49:11.561111 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.561002    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-openshift-service-ca\") pod \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\" (UID: \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\") "
Sep 06 10:49:11.561111 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.561027    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xm78\" (UniqueName: \"kubernetes.io/projected/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-kube-api-access-7xm78\") pod \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\" (UID: \"3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac\") "
Sep 06 10:49:11.561380 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.561333    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac" (UID: "3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:49:11.563637 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.563605    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac" (UID: "3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 10:49:11.563753 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.563685    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac" (UID: "3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 10:49:11.563753 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.563714    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-kube-api-access-7xm78" (OuterVolumeSpecName: "kube-api-access-7xm78") pod "3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac" (UID: "3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac"). InnerVolumeSpecName "kube-api-access-7xm78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:49:11.663003 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.662968    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-maas-api-tls\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:49:11.663003 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.663001    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-metrics-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:49:11.663204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.663015    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-openshift-service-ca\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:49:11.663204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.663029    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xm78\" (UniqueName: \"kubernetes.io/projected/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac-kube-api-access-7xm78\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:49:11.735020 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.734993    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"]
Sep 06 10:49:11.738804 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:11.738782    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aec3c2bf-68995896fd-tzclr"]
Sep 06 10:49:12.037875 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:12.037797    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac" path="/var/lib/kubelet/pods/3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac/volumes"
Sep 06 10:49:19.122237 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:19.122201    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:49:36.519676 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:36.519639    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:49:48.092724 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:48.092692    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/2.log"
Sep 06 10:49:48.095946 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:48.095917    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/2.log"
Sep 06 10:49:48.241162 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:48.241129    2557 scope.go:117] "RemoveContainer" containerID="e5ee7a13cf7570dd22dfdafacf630eaa8c57cd260bd33be39e10ea409a725fe3"
Sep 06 10:49:48.249837 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:48.249816    2557 scope.go:117] "RemoveContainer" containerID="7c6e82be54cec680ddd1702a46934934a4c3c7e9b3ff3a900d98dbb44715e0dc"
Sep 06 10:49:48.257715 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:48.257683    2557 scope.go:117] "RemoveContainer" containerID="372aed3c43494df98f80c682122091b536e6734ed94154f4c28123daff288027"
Sep 06 10:49:53.712737 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:49:53.712706    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:50:02.240514 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:02.240453    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"]
Sep 06 10:50:02.240949 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:02.240782    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q" podUID="0e635cc2-998d-4b39-ba2a-db7501658996" containerName="istio-proxy" containerID="cri-o://b08efdb2174e76dba65842c38c37dc2518b033d86c4ee37edfa3a2d342e19794" gracePeriod=30
Sep 06 10:50:07.505261 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.505205    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:50:07.690501 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.690462    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-istio-envoy\") pod \"0e635cc2-998d-4b39-ba2a-db7501658996\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") "
Sep 06 10:50:07.690677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.690556    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e635cc2-998d-4b39-ba2a-db7501658996-istio-token\") pod \"0e635cc2-998d-4b39-ba2a-db7501658996\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") "
Sep 06 10:50:07.690677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.690620    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-istio-data\") pod \"0e635cc2-998d-4b39-ba2a-db7501658996\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") "
Sep 06 10:50:07.690677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.690638    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-workload-socket\") pod \"0e635cc2-998d-4b39-ba2a-db7501658996\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") "
Sep 06 10:50:07.690677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.690665    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-credential-socket\") pod \"0e635cc2-998d-4b39-ba2a-db7501658996\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") "
Sep 06 10:50:07.690912 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.690722    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnlks\" (UniqueName: \"kubernetes.io/projected/0e635cc2-998d-4b39-ba2a-db7501658996-kube-api-access-xnlks\") pod \"0e635cc2-998d-4b39-ba2a-db7501658996\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") "
Sep 06 10:50:07.690912 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.690744    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e635cc2-998d-4b39-ba2a-db7501658996-istio-podinfo\") pod \"0e635cc2-998d-4b39-ba2a-db7501658996\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") "
Sep 06 10:50:07.690912 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.690789    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e635cc2-998d-4b39-ba2a-db7501658996-istiod-ca-cert\") pod \"0e635cc2-998d-4b39-ba2a-db7501658996\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") "
Sep 06 10:50:07.690912 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.690813    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-workload-certs\") pod \"0e635cc2-998d-4b39-ba2a-db7501658996\" (UID: \"0e635cc2-998d-4b39-ba2a-db7501658996\") "
Sep 06 10:50:07.691105 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.690932    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0e635cc2-998d-4b39-ba2a-db7501658996" (UID: "0e635cc2-998d-4b39-ba2a-db7501658996"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:50:07.691105 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.690928    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-istio-data" (OuterVolumeSpecName: "istio-data") pod "0e635cc2-998d-4b39-ba2a-db7501658996" (UID: "0e635cc2-998d-4b39-ba2a-db7501658996"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:50:07.691206 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.691129    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-istio-data\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:50:07.691206 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.691150    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-workload-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:50:07.691321 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.691299    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0e635cc2-998d-4b39-ba2a-db7501658996" (UID: "0e635cc2-998d-4b39-ba2a-db7501658996"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:50:07.691396 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.691341    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e635cc2-998d-4b39-ba2a-db7501658996-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0e635cc2-998d-4b39-ba2a-db7501658996" (UID: "0e635cc2-998d-4b39-ba2a-db7501658996"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:50:07.691460 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.691405    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0e635cc2-998d-4b39-ba2a-db7501658996" (UID: "0e635cc2-998d-4b39-ba2a-db7501658996"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:50:07.692704 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.692663    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0e635cc2-998d-4b39-ba2a-db7501658996" (UID: "0e635cc2-998d-4b39-ba2a-db7501658996"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:50:07.692828 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.692760    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e635cc2-998d-4b39-ba2a-db7501658996-istio-token" (OuterVolumeSpecName: "istio-token") pod "0e635cc2-998d-4b39-ba2a-db7501658996" (UID: "0e635cc2-998d-4b39-ba2a-db7501658996"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:50:07.693182 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.693164    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e635cc2-998d-4b39-ba2a-db7501658996-kube-api-access-xnlks" (OuterVolumeSpecName: "kube-api-access-xnlks") pod "0e635cc2-998d-4b39-ba2a-db7501658996" (UID: "0e635cc2-998d-4b39-ba2a-db7501658996"). InnerVolumeSpecName "kube-api-access-xnlks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:50:07.693240 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.693200    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e635cc2-998d-4b39-ba2a-db7501658996-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0e635cc2-998d-4b39-ba2a-db7501658996" (UID: "0e635cc2-998d-4b39-ba2a-db7501658996"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 10:50:07.744130 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.744087    2557 generic.go:356] "Generic (PLEG): container finished" podID="0e635cc2-998d-4b39-ba2a-db7501658996" containerID="b08efdb2174e76dba65842c38c37dc2518b033d86c4ee37edfa3a2d342e19794" exitCode=0
Sep 06 10:50:07.744286 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.744136    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"
Sep 06 10:50:07.744286 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.744178    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q" event={"ID":"0e635cc2-998d-4b39-ba2a-db7501658996","Type":"ContainerDied","Data":"b08efdb2174e76dba65842c38c37dc2518b033d86c4ee37edfa3a2d342e19794"}
Sep 06 10:50:07.744286 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.744234    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q" event={"ID":"0e635cc2-998d-4b39-ba2a-db7501658996","Type":"ContainerDied","Data":"0b692b2babfa5a68c5fd33d26662e23fcd38da45e38d0c6fd7ddb72845bd9483"}
Sep 06 10:50:07.744286 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.744260    2557 scope.go:117] "RemoveContainer" containerID="b08efdb2174e76dba65842c38c37dc2518b033d86c4ee37edfa3a2d342e19794"
Sep 06 10:50:07.754171 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.754151    2557 scope.go:117] "RemoveContainer" containerID="327be180c07fb48a892651421d913d90a1a89cd6af64b4572c87d68c218d30bd"
Sep 06 10:50:07.762750 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.762691    2557 scope.go:117] "RemoveContainer" containerID="b08efdb2174e76dba65842c38c37dc2518b033d86c4ee37edfa3a2d342e19794"
Sep 06 10:50:07.763135 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:50:07.763104    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b08efdb2174e76dba65842c38c37dc2518b033d86c4ee37edfa3a2d342e19794\": container with ID starting with b08efdb2174e76dba65842c38c37dc2518b033d86c4ee37edfa3a2d342e19794 not found: ID does not exist" containerID="b08efdb2174e76dba65842c38c37dc2518b033d86c4ee37edfa3a2d342e19794"
Sep 06 10:50:07.763271 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.763156    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b08efdb2174e76dba65842c38c37dc2518b033d86c4ee37edfa3a2d342e19794"} err="failed to get container status \"b08efdb2174e76dba65842c38c37dc2518b033d86c4ee37edfa3a2d342e19794\": rpc error: code = NotFound desc = could not find container \"b08efdb2174e76dba65842c38c37dc2518b033d86c4ee37edfa3a2d342e19794\": container with ID starting with b08efdb2174e76dba65842c38c37dc2518b033d86c4ee37edfa3a2d342e19794 not found: ID does not exist"
Sep 06 10:50:07.763271 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.763179    2557 scope.go:117] "RemoveContainer" containerID="327be180c07fb48a892651421d913d90a1a89cd6af64b4572c87d68c218d30bd"
Sep 06 10:50:07.763559 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:50:07.763534    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"327be180c07fb48a892651421d913d90a1a89cd6af64b4572c87d68c218d30bd\": container with ID starting with 327be180c07fb48a892651421d913d90a1a89cd6af64b4572c87d68c218d30bd not found: ID does not exist" containerID="327be180c07fb48a892651421d913d90a1a89cd6af64b4572c87d68c218d30bd"
Sep 06 10:50:07.763641 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.763566    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"327be180c07fb48a892651421d913d90a1a89cd6af64b4572c87d68c218d30bd"} err="failed to get container status \"327be180c07fb48a892651421d913d90a1a89cd6af64b4572c87d68c218d30bd\": rpc error: code = NotFound desc = could not find container \"327be180c07fb48a892651421d913d90a1a89cd6af64b4572c87d68c218d30bd\": container with ID starting with 327be180c07fb48a892651421d913d90a1a89cd6af64b4572c87d68c218d30bd not found: ID does not exist"
Sep 06 10:50:07.765697 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.765673    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"]
Sep 06 10:50:07.768969 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.768938    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-aec3c2bf-openshift-default-fdf97f656-c8c9q"]
Sep 06 10:50:07.791851 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.791818    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnlks\" (UniqueName: \"kubernetes.io/projected/0e635cc2-998d-4b39-ba2a-db7501658996-kube-api-access-xnlks\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:50:07.791851 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.791844    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e635cc2-998d-4b39-ba2a-db7501658996-istio-podinfo\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:50:07.791851 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.791856    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e635cc2-998d-4b39-ba2a-db7501658996-istiod-ca-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:50:07.792070 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.791865    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-workload-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:50:07.792070 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.791873    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-istio-envoy\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:50:07.792070 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.791880    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e635cc2-998d-4b39-ba2a-db7501658996-istio-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:50:07.792070 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:07.791888    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e635cc2-998d-4b39-ba2a-db7501658996-credential-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:50:08.036823 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:08.036741    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e635cc2-998d-4b39-ba2a-db7501658996" path="/var/lib/kubelet/pods/0e635cc2-998d-4b39-ba2a-db7501658996/volumes"
Sep 06 10:50:10.018169 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:10.018138    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:50:21.258834 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:21.258798    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7820-43c379"]
Sep 06 10:50:21.259293 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:21.259177    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac" containerName="maas-api"
Sep 06 10:50:21.259293 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:21.259189    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac" containerName="maas-api"
Sep 06 10:50:21.259293 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:21.259211    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e635cc2-998d-4b39-ba2a-db7501658996" containerName="istio-proxy"
Sep 06 10:50:21.259293 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:21.259218    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e635cc2-998d-4b39-ba2a-db7501658996" containerName="istio-proxy"
Sep 06 10:50:21.259293 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:21.259226    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e635cc2-998d-4b39-ba2a-db7501658996" containerName="istio-proxy"
Sep 06 10:50:21.259293 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:21.259231    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e635cc2-998d-4b39-ba2a-db7501658996" containerName="istio-proxy"
Sep 06 10:50:21.259293 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:21.259289    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ca59ad2-7f5f-44b6-9084-f2bc6c28e6ac" containerName="maas-api"
Sep 06 10:50:21.259293 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:21.259297    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e635cc2-998d-4b39-ba2a-db7501658996" containerName="istio-proxy"
Sep 06 10:50:21.262048 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:21.262030    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7820-43c379"
Sep 06 10:50:21.264506 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:21.264471    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lwnht\""
Sep 06 10:50:21.266678 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:21.266650    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7820-43c379"]
Sep 06 10:50:21.319555 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:21.319517    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4xsc\" (UniqueName: \"kubernetes.io/projected/9654f859-586a-4c21-8bf5-dc16b805f2d1-kube-api-access-m4xsc\") pod \"mt-curl-7820-43c379\" (UID: \"9654f859-586a-4c21-8bf5-dc16b805f2d1\") " pod="opendatahub/mt-curl-7820-43c379"
Sep 06 10:50:21.421081 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:21.421048    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4xsc\" (UniqueName: \"kubernetes.io/projected/9654f859-586a-4c21-8bf5-dc16b805f2d1-kube-api-access-m4xsc\") pod \"mt-curl-7820-43c379\" (UID: \"9654f859-586a-4c21-8bf5-dc16b805f2d1\") " pod="opendatahub/mt-curl-7820-43c379"
Sep 06 10:50:21.428557 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:21.428520    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4xsc\" (UniqueName: \"kubernetes.io/projected/9654f859-586a-4c21-8bf5-dc16b805f2d1-kube-api-access-m4xsc\") pod \"mt-curl-7820-43c379\" (UID: \"9654f859-586a-4c21-8bf5-dc16b805f2d1\") " pod="opendatahub/mt-curl-7820-43c379"
Sep 06 10:50:21.573553 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:21.573469    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7820-43c379"
Sep 06 10:50:21.849064 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:21.849037    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7820-43c379"]
Sep 06 10:50:21.851100 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:50:21.851073    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9654f859_586a_4c21_8bf5_dc16b805f2d1.slice/crio-6fe2b72e8b34bf7b87e336625d29dcf5c704ee2a250e63d2ba31fdd67830163c WatchSource:0}: Error finding container 6fe2b72e8b34bf7b87e336625d29dcf5c704ee2a250e63d2ba31fdd67830163c: Status 404 returned error can't find the container with id 6fe2b72e8b34bf7b87e336625d29dcf5c704ee2a250e63d2ba31fdd67830163c
Sep 06 10:50:22.832084 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:22.831989    2557 generic.go:356] "Generic (PLEG): container finished" podID="9654f859-586a-4c21-8bf5-dc16b805f2d1" containerID="4f3f62c56b690813e92b8c18cf418887851e3a0731d09976f3cb1c2afed4e2a7" exitCode=0
Sep 06 10:50:22.832424 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:22.832078    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7820-43c379" event={"ID":"9654f859-586a-4c21-8bf5-dc16b805f2d1","Type":"ContainerDied","Data":"4f3f62c56b690813e92b8c18cf418887851e3a0731d09976f3cb1c2afed4e2a7"}
Sep 06 10:50:22.832424 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:22.832118    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7820-43c379" event={"ID":"9654f859-586a-4c21-8bf5-dc16b805f2d1","Type":"ContainerStarted","Data":"6fe2b72e8b34bf7b87e336625d29dcf5c704ee2a250e63d2ba31fdd67830163c"}
Sep 06 10:50:23.981602 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:23.981580    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7820-43c379"
Sep 06 10:50:24.013654 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:24.013627    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7820-43c379_9654f859-586a-4c21-8bf5-dc16b805f2d1/mt-curl-7820-43c379/0.log"
Sep 06 10:50:24.039147 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:24.039112    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7820-43c379"]
Sep 06 10:50:24.041070 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:24.041049    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7820-43c379"]
Sep 06 10:50:24.043195 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:24.043177    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4xsc\" (UniqueName: \"kubernetes.io/projected/9654f859-586a-4c21-8bf5-dc16b805f2d1-kube-api-access-m4xsc\") pod \"9654f859-586a-4c21-8bf5-dc16b805f2d1\" (UID: \"9654f859-586a-4c21-8bf5-dc16b805f2d1\") "
Sep 06 10:50:24.045242 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:24.045218    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9654f859-586a-4c21-8bf5-dc16b805f2d1-kube-api-access-m4xsc" (OuterVolumeSpecName: "kube-api-access-m4xsc") pod "9654f859-586a-4c21-8bf5-dc16b805f2d1" (UID: "9654f859-586a-4c21-8bf5-dc16b805f2d1"). InnerVolumeSpecName "kube-api-access-m4xsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:50:24.144887 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:24.144804    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4xsc\" (UniqueName: \"kubernetes.io/projected/9654f859-586a-4c21-8bf5-dc16b805f2d1-kube-api-access-m4xsc\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:50:24.716378 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:24.716341    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:50:24.845244 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:24.845215    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7820-43c379"
Sep 06 10:50:24.845244 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:24.845232    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6fe2b72e8b34bf7b87e336625d29dcf5c704ee2a250e63d2ba31fdd67830163c"
Sep 06 10:50:24.855169 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:50:24.855133    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7820-43c379\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="9654f859-586a-4c21-8bf5-dc16b805f2d1" pod="opendatahub/mt-curl-7820-43c379"
Sep 06 10:50:26.037220 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:26.037174    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9654f859-586a-4c21-8bf5-dc16b805f2d1" path="/var/lib/kubelet/pods/9654f859-586a-4c21-8bf5-dc16b805f2d1/volumes"
Sep 06 10:50:36.921310 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:36.921271    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:50:48.290048 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:48.289979    2557 scope.go:117] "RemoveContainer" containerID="bc0dcf7195f69916ab5cf99eb3bd8e3d2e03b7a5180c6f9aab72116f6f8dd1a4"
Sep 06 10:50:50.613882 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:50.613847    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:50:55.967315 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:55.967280    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"]
Sep 06 10:50:55.968034 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:55.968011    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9654f859-586a-4c21-8bf5-dc16b805f2d1" containerName="mt-curl-7820-43c379"
Sep 06 10:50:55.968136 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:55.968037    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9654f859-586a-4c21-8bf5-dc16b805f2d1" containerName="mt-curl-7820-43c379"
Sep 06 10:50:55.968204 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:55.968186    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9654f859-586a-4c21-8bf5-dc16b805f2d1" containerName="mt-curl-7820-43c379"
Sep 06 10:50:55.968258 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:55.968205    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e635cc2-998d-4b39-ba2a-db7501658996" containerName="istio-proxy"
Sep 06 10:50:55.972814 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:55.972792    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:55.975160 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:55.975135    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-866a8b-openshift-default-dockercfg-sd7pw\""
Sep 06 10:50:55.982888 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:55.982852    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"]
Sep 06 10:50:56.042185 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.042157    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-token\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.042339 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.042201    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-workload-socket\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.042339 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.042221    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-credential-socket\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.042339 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.042290    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wftth\" (UniqueName: \"kubernetes.io/projected/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-kube-api-access-wftth\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.042339 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.042330    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istiod-ca-cert\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.042582 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.042386    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-workload-certs\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.042582 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.042408    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-data\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.042582 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.042463    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-envoy\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.042582 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.042483    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-podinfo\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.143419 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.143383    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-workload-certs\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.143624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.143430    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-data\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.143624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.143485    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-envoy\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.143624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.143548    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-podinfo\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.143624 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.143597    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-token\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.144056 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.143643    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-workload-socket\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.144056 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.143673    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-credential-socket\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.144056 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.143707    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wftth\" (UniqueName: \"kubernetes.io/projected/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-kube-api-access-wftth\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.144056 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.143744    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istiod-ca-cert\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.144056 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.143860    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-workload-certs\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.144056 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.143898    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-data\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.144056 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.143964    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-workload-socket\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.144056 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.144012    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-credential-socket\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.144552 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.144520    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istiod-ca-cert\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.146083 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.146057    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-envoy\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.146193 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.146107    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-podinfo\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.151443 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.151421    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-token\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.151681 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.151663    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wftth\" (UniqueName: \"kubernetes.io/projected/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-kube-api-access-wftth\") pod \"e2e-api-a-866a8b-openshift-default-585855d566-lpffv\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") " pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.286016 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.285925    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:56.559724 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.559699    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"]
Sep 06 10:50:56.561171 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:50:56.561146    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddceb8afc_b5fe_41ea_822e_d60a1524c7b8.slice/crio-fc5aa256130ef92079920d94c10e21a63df9dec469aec12bfa71691c1eb96572 WatchSource:0}: Error finding container fc5aa256130ef92079920d94c10e21a63df9dec469aec12bfa71691c1eb96572: Status 404 returned error can't find the container with id fc5aa256130ef92079920d94c10e21a63df9dec469aec12bfa71691c1eb96572
Sep 06 10:50:56.563548 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.563512    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:50:56.563659 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.563577    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:50:56.563659 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:56.563612    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:50:57.029308 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:57.029268    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv" event={"ID":"dceb8afc-b5fe-41ea-822e-d60a1524c7b8","Type":"ContainerStarted","Data":"b6837a8f7acb2c8e17751c022ac21bcb6f745ef85c3bb9b28335c50dc73e13ab"}
Sep 06 10:50:57.029794 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:57.029353    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv" event={"ID":"dceb8afc-b5fe-41ea-822e-d60a1524c7b8","Type":"ContainerStarted","Data":"fc5aa256130ef92079920d94c10e21a63df9dec469aec12bfa71691c1eb96572"}
Sep 06 10:50:57.048274 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:57.048226    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv" podStartSLOduration=2.048208991 podStartE2EDuration="2.048208991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:50:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:50:57.047078057 +0000 UTC m=+1269.496858826" watchObservedRunningTime="2026-09-06 10:50:57.048208991 +0000 UTC m=+1269.497989760"
Sep 06 10:50:57.287004 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:57.286916    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:57.291729 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:57.291708    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:58.036436 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:58.036409    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:50:58.036812 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:50:58.036466    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:51:00.054846 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.054809    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"]
Sep 06 10:51:00.059205 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.059184    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"
Sep 06 10:51:00.061641 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.061617    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 06 10:51:00.062987 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.062962    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 06 10:51:00.063313 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.063295    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 10:51:00.063568 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.063462    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q972h\""
Sep 06 10:51:00.063845 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.063470    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-866a8b-serving-cert\""
Sep 06 10:51:00.071535 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.071081    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"]
Sep 06 10:51:00.081721 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.081699    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d12595e-1d5f-41b5-8382-0331e41279f5-metrics-certs\") pod \"maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj\" (UID: \"7d12595e-1d5f-41b5-8382-0331e41279f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"
Sep 06 10:51:00.081832 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.081755    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d12595e-1d5f-41b5-8382-0331e41279f5-maas-api-tls\") pod \"maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj\" (UID: \"7d12595e-1d5f-41b5-8382-0331e41279f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"
Sep 06 10:51:00.081832 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.081777    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzz44\" (UniqueName: \"kubernetes.io/projected/7d12595e-1d5f-41b5-8382-0331e41279f5-kube-api-access-qzz44\") pod \"maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj\" (UID: \"7d12595e-1d5f-41b5-8382-0331e41279f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"
Sep 06 10:51:00.081832 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.081810    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d12595e-1d5f-41b5-8382-0331e41279f5-openshift-service-ca\") pod \"maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj\" (UID: \"7d12595e-1d5f-41b5-8382-0331e41279f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"
Sep 06 10:51:00.183049 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.183007    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d12595e-1d5f-41b5-8382-0331e41279f5-metrics-certs\") pod \"maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj\" (UID: \"7d12595e-1d5f-41b5-8382-0331e41279f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"
Sep 06 10:51:00.183221 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.183068    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d12595e-1d5f-41b5-8382-0331e41279f5-maas-api-tls\") pod \"maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj\" (UID: \"7d12595e-1d5f-41b5-8382-0331e41279f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"
Sep 06 10:51:00.183221 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.183092    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzz44\" (UniqueName: \"kubernetes.io/projected/7d12595e-1d5f-41b5-8382-0331e41279f5-kube-api-access-qzz44\") pod \"maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj\" (UID: \"7d12595e-1d5f-41b5-8382-0331e41279f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"
Sep 06 10:51:00.183221 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.183113    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d12595e-1d5f-41b5-8382-0331e41279f5-openshift-service-ca\") pod \"maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj\" (UID: \"7d12595e-1d5f-41b5-8382-0331e41279f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"
Sep 06 10:51:00.183324 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:51:00.183238    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-serving-cert: secret "maas-api-e2e-api-a-866a8b-serving-cert" not found
Sep 06 10:51:00.183368 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:51:00.183323    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d12595e-1d5f-41b5-8382-0331e41279f5-maas-api-tls podName:7d12595e-1d5f-41b5-8382-0331e41279f5 nodeName:}" failed. No retries permitted until 2026-09-06 10:51:00.683300573 +0000 UTC m=+1273.133081323 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7d12595e-1d5f-41b5-8382-0331e41279f5-maas-api-tls") pod "maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj" (UID: "7d12595e-1d5f-41b5-8382-0331e41279f5") : secret "maas-api-e2e-api-a-866a8b-serving-cert" not found
Sep 06 10:51:00.183910 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.183884    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d12595e-1d5f-41b5-8382-0331e41279f5-openshift-service-ca\") pod \"maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj\" (UID: \"7d12595e-1d5f-41b5-8382-0331e41279f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"
Sep 06 10:51:00.185522 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.185483    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d12595e-1d5f-41b5-8382-0331e41279f5-metrics-certs\") pod \"maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj\" (UID: \"7d12595e-1d5f-41b5-8382-0331e41279f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"
Sep 06 10:51:00.198556 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.198529    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzz44\" (UniqueName: \"kubernetes.io/projected/7d12595e-1d5f-41b5-8382-0331e41279f5-kube-api-access-qzz44\") pod \"maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj\" (UID: \"7d12595e-1d5f-41b5-8382-0331e41279f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"
Sep 06 10:51:00.688041 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.688002    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d12595e-1d5f-41b5-8382-0331e41279f5-maas-api-tls\") pod \"maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj\" (UID: \"7d12595e-1d5f-41b5-8382-0331e41279f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"
Sep 06 10:51:00.691052 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.691023    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d12595e-1d5f-41b5-8382-0331e41279f5-maas-api-tls\") pod \"maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj\" (UID: \"7d12595e-1d5f-41b5-8382-0331e41279f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"
Sep 06 10:51:00.976111 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:00.976027    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"
Sep 06 10:51:01.245485 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:01.245456    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"]
Sep 06 10:51:01.247748 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:51:01.247719    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d12595e_1d5f_41b5_8382_0331e41279f5.slice/crio-87bef79ccae7a7bebbe114fb58993de793cc7feb06c8e76c92460dd99137de69 WatchSource:0}: Error finding container 87bef79ccae7a7bebbe114fb58993de793cc7feb06c8e76c92460dd99137de69: Status 404 returned error can't find the container with id 87bef79ccae7a7bebbe114fb58993de793cc7feb06c8e76c92460dd99137de69
Sep 06 10:51:02.065099 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:02.065015    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj" event={"ID":"7d12595e-1d5f-41b5-8382-0331e41279f5","Type":"ContainerStarted","Data":"b777acfaf3258fa0b626b65c1defaea477e6e24faa867fcad420217e0958bdf3"}
Sep 06 10:51:02.065099 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:02.065078    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj" event={"ID":"7d12595e-1d5f-41b5-8382-0331e41279f5","Type":"ContainerStarted","Data":"87bef79ccae7a7bebbe114fb58993de793cc7feb06c8e76c92460dd99137de69"}
Sep 06 10:51:02.065478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:02.065428    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"
Sep 06 10:51:08.076899 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:08.076873    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"
Sep 06 10:51:08.093244 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:08.093201    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj" podStartSLOduration=7.659903524 podStartE2EDuration="8.093186373s" totalImagesPullingTime="433.282849ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:51:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:51:02.084803308 +0000 UTC m=+1274.534584122" watchObservedRunningTime="2026-09-06 10:51:08.093186373 +0000 UTC m=+1280.542967134"
Sep 06 10:51:09.228081 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:09.228050    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:51:19.167686 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:19.167653    2557 generic.go:356] "Generic (PLEG): container finished" podID="dceb8afc-b5fe-41ea-822e-d60a1524c7b8" containerID="b6837a8f7acb2c8e17751c022ac21bcb6f745ef85c3bb9b28335c50dc73e13ab" exitCode=0
Sep 06 10:51:19.168218 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:19.167707    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv" event={"ID":"dceb8afc-b5fe-41ea-822e-d60a1524c7b8","Type":"ContainerDied","Data":"b6837a8f7acb2c8e17751c022ac21bcb6f745ef85c3bb9b28335c50dc73e13ab"}
Sep 06 10:51:19.168218 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:19.168085    2557 scope.go:117] "RemoveContainer" containerID="b6837a8f7acb2c8e17751c022ac21bcb6f745ef85c3bb9b28335c50dc73e13ab"
Sep 06 10:51:19.169237 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:19.169203    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:51:19.169316 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:19.169278    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:51:19.169316 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:19.169309    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:51:20.183856 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:20.183817    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv" event={"ID":"dceb8afc-b5fe-41ea-822e-d60a1524c7b8","Type":"ContainerStarted","Data":"5fe80bf7097a28f24cc43260a270f878939f3ef93e83a3fb79570c5bf6fea67e"}
Sep 06 10:51:20.286967 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:20.286927    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:51:20.287178 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:20.287054    2557 patch_prober.go:28] interesting pod/e2e-api-a-866a8b-openshift-default-585855d566-lpffv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 06 10:51:20.287178 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:20.287098    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv" podUID="dceb8afc-b5fe-41ea-822e-d60a1524c7b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 06 10:51:21.286511 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:21.286453    2557 patch_prober.go:28] interesting pod/e2e-api-a-866a8b-openshift-default-585855d566-lpffv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 06 10:51:21.287104 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:21.287072    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv" podUID="dceb8afc-b5fe-41ea-822e-d60a1524c7b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 06 10:51:22.287060 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:22.287015    2557 patch_prober.go:28] interesting pod/e2e-api-a-866a8b-openshift-default-585855d566-lpffv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 06 10:51:22.287582 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:22.287094    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv" podUID="dceb8afc-b5fe-41ea-822e-d60a1524c7b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 06 10:51:23.287085 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:23.287044    2557 patch_prober.go:28] interesting pod/e2e-api-a-866a8b-openshift-default-585855d566-lpffv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 06 10:51:23.287714 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:23.287671    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv" podUID="dceb8afc-b5fe-41ea-822e-d60a1524c7b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 06 10:51:23.726505 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:23.726464    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:51:24.287410 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.287374    2557 patch_prober.go:28] interesting pod/e2e-api-a-866a8b-openshift-default-585855d566-lpffv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 06 10:51:24.287832 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.287443    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv" podUID="dceb8afc-b5fe-41ea-822e-d60a1524c7b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 06 10:51:24.692515 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.691968    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"]
Sep 06 10:51:24.718386 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.718355    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"]
Sep 06 10:51:24.718556 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.718464    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.720833 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.720815    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-b83cfe-openshift-default-dockercfg-mttbv\""
Sep 06 10:51:24.829892 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.829860    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-credential-socket\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.830092 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.829947    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-data\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.830092 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.830001    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-token\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.830092 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.830025    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-workload-certs\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.830092 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.830047    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-envoy\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.830092 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.830084    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istiod-ca-cert\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.830534 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.830129    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-podinfo\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.830534 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.830174    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfm9f\" (UniqueName: \"kubernetes.io/projected/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-kube-api-access-nfm9f\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.830534 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.830242    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-workload-socket\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.931454 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.931416    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-credential-socket\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.931685 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.931478    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-data\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.931685 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.931546    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-token\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.931685 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.931573    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-workload-certs\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.931685 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.931596    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-envoy\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.931685 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.931634    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istiod-ca-cert\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.931685 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.931674    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-podinfo\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.932023 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.931717    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfm9f\" (UniqueName: \"kubernetes.io/projected/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-kube-api-access-nfm9f\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.932023 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.931784    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-workload-socket\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.932023 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.931872    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-data\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.932023 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.931958    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-workload-certs\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.932252 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.932177    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-workload-socket\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.932252 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.932183    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-credential-socket\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.932418 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.932395    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istiod-ca-cert\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.936515 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.934017    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-envoy\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.936515 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.934244    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-podinfo\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.938883 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.938855    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-token\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:24.938984 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:24.938965    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfm9f\" (UniqueName: \"kubernetes.io/projected/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-kube-api-access-nfm9f\") pod \"e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") " pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:25.033161 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:25.033076    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:25.326809 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:25.326620    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"]
Sep 06 10:51:25.333800 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:25.333207    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:51:25.333800 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:25.333299    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:51:25.333800 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:25.333345    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:51:25.513544 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:25.513505    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:51:25.514901 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:25.514877    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:51:25.516571 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:25.516521    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:51:26.223191 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:26.223151    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4" event={"ID":"c3bebebf-c413-4169-b1e2-68fa22ca8ac7","Type":"ContainerStarted","Data":"6ff436e6867bca8d9b43e0cf7091478c915c25accdb1ed092318e9e6623ceddc"}
Sep 06 10:51:26.223378 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:26.223210    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4" event={"ID":"c3bebebf-c413-4169-b1e2-68fa22ca8ac7","Type":"ContainerStarted","Data":"75bebff4eb6f9c38a3eded63e81659c1ae1475cb6a2351aeb29721be146402db"}
Sep 06 10:51:26.244587 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:26.244546    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4" podStartSLOduration=2.244531466 podStartE2EDuration="2.244531466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:51:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:51:26.242104199 +0000 UTC m=+1298.691884967" watchObservedRunningTime="2026-09-06 10:51:26.244531466 +0000 UTC m=+1298.694312234"
Sep 06 10:51:27.033470 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:27.033432    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:27.039025 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:27.038995    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:27.229629 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:27.229597    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:27.230479 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:27.230456    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:29.245623 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:29.245587    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"]
Sep 06 10:51:29.250216 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:29.250192    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:51:29.252680 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:29.252642    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-b83cfe-serving-cert\""
Sep 06 10:51:29.256600 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:29.256577    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"]
Sep 06 10:51:29.278020 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:29.277982    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/845f246a-3957-4dde-8956-85a03dc8ecea-openshift-service-ca\") pod \"maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs\" (UID: \"845f246a-3957-4dde-8956-85a03dc8ecea\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:51:29.278227 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:29.278116    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845f246a-3957-4dde-8956-85a03dc8ecea-metrics-certs\") pod \"maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs\" (UID: \"845f246a-3957-4dde-8956-85a03dc8ecea\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:51:29.278227 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:29.278176    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9629h\" (UniqueName: \"kubernetes.io/projected/845f246a-3957-4dde-8956-85a03dc8ecea-kube-api-access-9629h\") pod \"maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs\" (UID: \"845f246a-3957-4dde-8956-85a03dc8ecea\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:51:29.278376 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:29.278245    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/845f246a-3957-4dde-8956-85a03dc8ecea-maas-api-tls\") pod \"maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs\" (UID: \"845f246a-3957-4dde-8956-85a03dc8ecea\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:51:29.379417 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:29.379384    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/845f246a-3957-4dde-8956-85a03dc8ecea-openshift-service-ca\") pod \"maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs\" (UID: \"845f246a-3957-4dde-8956-85a03dc8ecea\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:51:29.379628 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:29.379448    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845f246a-3957-4dde-8956-85a03dc8ecea-metrics-certs\") pod \"maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs\" (UID: \"845f246a-3957-4dde-8956-85a03dc8ecea\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:51:29.379628 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:29.379502    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9629h\" (UniqueName: \"kubernetes.io/projected/845f246a-3957-4dde-8956-85a03dc8ecea-kube-api-access-9629h\") pod \"maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs\" (UID: \"845f246a-3957-4dde-8956-85a03dc8ecea\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:51:29.379628 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:29.379550    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/845f246a-3957-4dde-8956-85a03dc8ecea-maas-api-tls\") pod \"maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs\" (UID: \"845f246a-3957-4dde-8956-85a03dc8ecea\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:51:29.379771 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:51:29.379680    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-serving-cert: secret "maas-api-e2e-api-b-b83cfe-serving-cert" not found
Sep 06 10:51:29.379771 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:51:29.379745    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/845f246a-3957-4dde-8956-85a03dc8ecea-maas-api-tls podName:845f246a-3957-4dde-8956-85a03dc8ecea nodeName:}" failed. No retries permitted until 2026-09-06 10:51:29.879726795 +0000 UTC m=+1302.329507553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/845f246a-3957-4dde-8956-85a03dc8ecea-maas-api-tls") pod "maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs" (UID: "845f246a-3957-4dde-8956-85a03dc8ecea") : secret "maas-api-e2e-api-b-b83cfe-serving-cert" not found
Sep 06 10:51:29.380216 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:29.380186    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/845f246a-3957-4dde-8956-85a03dc8ecea-openshift-service-ca\") pod \"maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs\" (UID: \"845f246a-3957-4dde-8956-85a03dc8ecea\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:51:29.382137 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:29.382109    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845f246a-3957-4dde-8956-85a03dc8ecea-metrics-certs\") pod \"maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs\" (UID: \"845f246a-3957-4dde-8956-85a03dc8ecea\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:51:29.397350 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:29.397321    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9629h\" (UniqueName: \"kubernetes.io/projected/845f246a-3957-4dde-8956-85a03dc8ecea-kube-api-access-9629h\") pod \"maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs\" (UID: \"845f246a-3957-4dde-8956-85a03dc8ecea\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:51:29.884170 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:29.884132    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/845f246a-3957-4dde-8956-85a03dc8ecea-maas-api-tls\") pod \"maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs\" (UID: \"845f246a-3957-4dde-8956-85a03dc8ecea\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:51:29.884364 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:51:29.884311    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-serving-cert: secret "maas-api-e2e-api-b-b83cfe-serving-cert" not found
Sep 06 10:51:29.884412 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:51:29.884393    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/845f246a-3957-4dde-8956-85a03dc8ecea-maas-api-tls podName:845f246a-3957-4dde-8956-85a03dc8ecea nodeName:}" failed. No retries permitted until 2026-09-06 10:51:30.884373537 +0000 UTC m=+1303.334154289 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/845f246a-3957-4dde-8956-85a03dc8ecea-maas-api-tls") pod "maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs" (UID: "845f246a-3957-4dde-8956-85a03dc8ecea") : secret "maas-api-e2e-api-b-b83cfe-serving-cert" not found
Sep 06 10:51:30.895954 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:30.895917    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/845f246a-3957-4dde-8956-85a03dc8ecea-maas-api-tls\") pod \"maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs\" (UID: \"845f246a-3957-4dde-8956-85a03dc8ecea\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:51:30.898422 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:30.898397    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/845f246a-3957-4dde-8956-85a03dc8ecea-maas-api-tls\") pod \"maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs\" (UID: \"845f246a-3957-4dde-8956-85a03dc8ecea\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:51:31.065336 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:31.065293    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:51:31.344462 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:31.344433    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"]
Sep 06 10:51:31.345757 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:51:31.345734    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod845f246a_3957_4dde_8956_85a03dc8ecea.slice/crio-63f10568a70a784b2c3a77b6a11aaacf82667b19478f760f3b52ae4a999dc7a3 WatchSource:0}: Error finding container 63f10568a70a784b2c3a77b6a11aaacf82667b19478f760f3b52ae4a999dc7a3: Status 404 returned error can't find the container with id 63f10568a70a784b2c3a77b6a11aaacf82667b19478f760f3b52ae4a999dc7a3
Sep 06 10:51:32.262302 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:32.262268    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs" event={"ID":"845f246a-3957-4dde-8956-85a03dc8ecea","Type":"ContainerStarted","Data":"b5e255358dc3d4edd8a31ec2bac24dc3060b4eb89627a68bbd7f394739db71a0"}
Sep 06 10:51:32.262733 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:32.262333    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs" event={"ID":"845f246a-3957-4dde-8956-85a03dc8ecea","Type":"ContainerStarted","Data":"63f10568a70a784b2c3a77b6a11aaacf82667b19478f760f3b52ae4a999dc7a3"}
Sep 06 10:51:32.262733 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:32.262391    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:51:32.277803 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:32.277756    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs" podStartSLOduration=2.983180196 podStartE2EDuration="3.277738319s" totalImagesPullingTime="294.558123ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:51:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:51:32.277236664 +0000 UTC m=+1304.727017433" watchObservedRunningTime="2026-09-06 10:51:32.277738319 +0000 UTC m=+1304.727519141"
Sep 06 10:51:38.221043 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:38.221006    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:51:38.274841 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:38.274816    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:51:48.316312 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:48.316268    2557 scope.go:117] "RemoveContainer" containerID="0403895c3bb30ad78ae0eeeed60d7eec0d7665d57f92b9a9609ea1353a391193"
Sep 06 10:51:48.329122 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:48.329101    2557 scope.go:117] "RemoveContainer" containerID="98c8479fea72f8fe7b57bf8cb77ea21943abebb7f23981d3b14e2ffe59caed44"
Sep 06 10:51:48.337177 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:48.337154    2557 scope.go:117] "RemoveContainer" containerID="e244e2dce5a9142bbaafb05add6fa09755b505773b1f13e782ae2a8270fa4bcb"
Sep 06 10:51:48.344990 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:48.344969    2557 scope.go:117] "RemoveContainer" containerID="6494440d9a6013c174965291894b801e882a23f39f2e9836ec37b8d115b1216b"
Sep 06 10:51:48.352975 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:48.352954    2557 scope.go:117] "RemoveContainer" containerID="1ba27cb1b73d3e648efbe541bae733bbe304168238c71b3bdb37a2b5fb7bf4f7"
Sep 06 10:51:49.368808 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:49.368775    2557 generic.go:356] "Generic (PLEG): container finished" podID="c3bebebf-c413-4169-b1e2-68fa22ca8ac7" containerID="6ff436e6867bca8d9b43e0cf7091478c915c25accdb1ed092318e9e6623ceddc" exitCode=0
Sep 06 10:51:49.369258 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:49.368848    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4" event={"ID":"c3bebebf-c413-4169-b1e2-68fa22ca8ac7","Type":"ContainerDied","Data":"6ff436e6867bca8d9b43e0cf7091478c915c25accdb1ed092318e9e6623ceddc"}
Sep 06 10:51:49.369311 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:49.369294    2557 scope.go:117] "RemoveContainer" containerID="6ff436e6867bca8d9b43e0cf7091478c915c25accdb1ed092318e9e6623ceddc"
Sep 06 10:51:49.370399 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:49.370362    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:51:49.370508 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:49.370435    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:51:49.370508 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:49.370465    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:51:50.402281 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:50.402236    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4" event={"ID":"c3bebebf-c413-4169-b1e2-68fa22ca8ac7","Type":"ContainerStarted","Data":"ac34c6755911550b2231da9251344fe9ccefbfd504626c17ebd38614ac1c15f6"}
Sep 06 10:51:51.034177 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:51.034138    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:51.034374 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:51.034327    2557 patch_prober.go:28] interesting pod/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 06 10:51:51.034424 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:51.034374    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4" podUID="c3bebebf-c413-4169-b1e2-68fa22ca8ac7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 06 10:51:52.034172 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:52.034142    2557 patch_prober.go:28] interesting pod/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 06 10:51:52.034595 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:52.034208    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4" podUID="c3bebebf-c413-4169-b1e2-68fa22ca8ac7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 06 10:51:53.033983 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:53.033946    2557 patch_prober.go:28] interesting pod/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 06 10:51:53.034164 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:53.034007    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4" podUID="c3bebebf-c413-4169-b1e2-68fa22ca8ac7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 06 10:51:54.033785 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:54.033753    2557 patch_prober.go:28] interesting pod/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 06 10:51:54.034169 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:54.033802    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4" podUID="c3bebebf-c413-4169-b1e2-68fa22ca8ac7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 06 10:51:55.033979 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:55.033949    2557 patch_prober.go:28] interesting pod/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 06 10:51:55.034382 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:55.034011    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4" podUID="c3bebebf-c413-4169-b1e2-68fa22ca8ac7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 06 10:51:56.039929 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:56.039900    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:56.040382 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:56.040296    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:56.040936 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:56.040917    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:51:58.320523 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:51:58.320464    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:52:00.721263 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:00.721225    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7820-afbf95"]
Sep 06 10:52:00.726108 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:00.726085    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7820-afbf95"
Sep 06 10:52:00.728298 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:00.728272    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lwnht\""
Sep 06 10:52:00.728481 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:00.728301    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7820-afbf95"]
Sep 06 10:52:00.900836 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:00.900797    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlxrw\" (UniqueName: \"kubernetes.io/projected/bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1-kube-api-access-tlxrw\") pod \"mt-curl-7820-afbf95\" (UID: \"bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1\") " pod="opendatahub/mt-curl-7820-afbf95"
Sep 06 10:52:01.002332 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:01.002240    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlxrw\" (UniqueName: \"kubernetes.io/projected/bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1-kube-api-access-tlxrw\") pod \"mt-curl-7820-afbf95\" (UID: \"bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1\") " pod="opendatahub/mt-curl-7820-afbf95"
Sep 06 10:52:01.010235 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:01.010205    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlxrw\" (UniqueName: \"kubernetes.io/projected/bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1-kube-api-access-tlxrw\") pod \"mt-curl-7820-afbf95\" (UID: \"bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1\") " pod="opendatahub/mt-curl-7820-afbf95"
Sep 06 10:52:01.037782 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:01.037758    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7820-afbf95"
Sep 06 10:52:01.307201 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:01.307175    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7820-afbf95"]
Sep 06 10:52:01.309270 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:52:01.309242    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc3d089b_c8b7_4f3d_98c1_fa2fc9f3cca1.slice/crio-bfd6273f1bc80982a5a6efc7fb3c3f4ab08c1d0efdaafb6d16b5e402fa9f7789 WatchSource:0}: Error finding container bfd6273f1bc80982a5a6efc7fb3c3f4ab08c1d0efdaafb6d16b5e402fa9f7789: Status 404 returned error can't find the container with id bfd6273f1bc80982a5a6efc7fb3c3f4ab08c1d0efdaafb6d16b5e402fa9f7789
Sep 06 10:52:01.465149 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:01.465115    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7820-afbf95" event={"ID":"bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1","Type":"ContainerStarted","Data":"bfd6273f1bc80982a5a6efc7fb3c3f4ab08c1d0efdaafb6d16b5e402fa9f7789"}
Sep 06 10:52:02.472507 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:02.472455    2557 generic.go:356] "Generic (PLEG): container finished" podID="bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1" containerID="c5b9fcb96a7a0319cc804ff4b5744e5955220f68f4ea22f0e3934fae3e2ab6c4" exitCode=0
Sep 06 10:52:02.472878 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:02.472525    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7820-afbf95" event={"ID":"bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1","Type":"ContainerDied","Data":"c5b9fcb96a7a0319cc804ff4b5744e5955220f68f4ea22f0e3934fae3e2ab6c4"}
Sep 06 10:52:03.618258 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:03.618231    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7820-afbf95"
Sep 06 10:52:03.628931 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:03.628911    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlxrw\" (UniqueName: \"kubernetes.io/projected/bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1-kube-api-access-tlxrw\") pod \"bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1\" (UID: \"bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1\") "
Sep 06 10:52:03.630812 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:03.630776    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1-kube-api-access-tlxrw" (OuterVolumeSpecName: "kube-api-access-tlxrw") pod "bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1" (UID: "bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1"). InnerVolumeSpecName "kube-api-access-tlxrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:52:03.649423 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:03.649402    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7820-afbf95_bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1/mt-curl-7820-afbf95/0.log"
Sep 06 10:52:03.673412 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:03.673388    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7820-afbf95"]
Sep 06 10:52:03.677378 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:03.677350    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7820-afbf95"]
Sep 06 10:52:03.730341 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:03.730267    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlxrw\" (UniqueName: \"kubernetes.io/projected/bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1-kube-api-access-tlxrw\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:03.789966 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:03.789942    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7820-5b6f07"]
Sep 06 10:52:03.790363 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:03.790349    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1" containerName="mt-curl-7820-afbf95"
Sep 06 10:52:03.790410 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:03.790365    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1" containerName="mt-curl-7820-afbf95"
Sep 06 10:52:03.790461 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:03.790452    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1" containerName="mt-curl-7820-afbf95"
Sep 06 10:52:03.793431 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:03.793417    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7820-5b6f07"
Sep 06 10:52:03.798549 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:03.798521    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7820-5b6f07"]
Sep 06 10:52:03.830822 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:03.830790    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2x7j\" (UniqueName: \"kubernetes.io/projected/5d715cb1-6e33-46c2-b25d-9219d152d70d-kube-api-access-d2x7j\") pod \"mt-curl-7820-5b6f07\" (UID: \"5d715cb1-6e33-46c2-b25d-9219d152d70d\") " pod="opendatahub/mt-curl-7820-5b6f07"
Sep 06 10:52:03.931783 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:03.931754    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2x7j\" (UniqueName: \"kubernetes.io/projected/5d715cb1-6e33-46c2-b25d-9219d152d70d-kube-api-access-d2x7j\") pod \"mt-curl-7820-5b6f07\" (UID: \"5d715cb1-6e33-46c2-b25d-9219d152d70d\") " pod="opendatahub/mt-curl-7820-5b6f07"
Sep 06 10:52:03.938768 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:03.938748    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2x7j\" (UniqueName: \"kubernetes.io/projected/5d715cb1-6e33-46c2-b25d-9219d152d70d-kube-api-access-d2x7j\") pod \"mt-curl-7820-5b6f07\" (UID: \"5d715cb1-6e33-46c2-b25d-9219d152d70d\") " pod="opendatahub/mt-curl-7820-5b6f07"
Sep 06 10:52:04.037198 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:04.037118    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1" path="/var/lib/kubelet/pods/bc3d089b-c8b7-4f3d-98c1-fa2fc9f3cca1/volumes"
Sep 06 10:52:04.105127 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:04.105095    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7820-5b6f07"
Sep 06 10:52:04.373615 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:04.373590    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7820-5b6f07"]
Sep 06 10:52:04.375396 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:52:04.375362    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d715cb1_6e33_46c2_b25d_9219d152d70d.slice/crio-2ff941bf876acc35f44e6e2c27f1c156013b6d886676d2a0a7a22d8ecf1cdbb2 WatchSource:0}: Error finding container 2ff941bf876acc35f44e6e2c27f1c156013b6d886676d2a0a7a22d8ecf1cdbb2: Status 404 returned error can't find the container with id 2ff941bf876acc35f44e6e2c27f1c156013b6d886676d2a0a7a22d8ecf1cdbb2
Sep 06 10:52:04.484406 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:04.484376    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7820-afbf95"
Sep 06 10:52:04.484586 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:04.484432    2557 scope.go:117] "RemoveContainer" containerID="c5b9fcb96a7a0319cc804ff4b5744e5955220f68f4ea22f0e3934fae3e2ab6c4"
Sep 06 10:52:04.487680 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:04.487404    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7820-5b6f07" event={"ID":"5d715cb1-6e33-46c2-b25d-9219d152d70d","Type":"ContainerStarted","Data":"2ff941bf876acc35f44e6e2c27f1c156013b6d886676d2a0a7a22d8ecf1cdbb2"}
Sep 06 10:52:05.494387 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:05.494357    2557 generic.go:356] "Generic (PLEG): container finished" podID="5d715cb1-6e33-46c2-b25d-9219d152d70d" containerID="a267b0d3e66130fa9b7a016f4f4691d5f0f2bfd799af16afd88335155c27c34a" exitCode=0
Sep 06 10:52:05.494823 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:05.494431    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7820-5b6f07" event={"ID":"5d715cb1-6e33-46c2-b25d-9219d152d70d","Type":"ContainerDied","Data":"a267b0d3e66130fa9b7a016f4f4691d5f0f2bfd799af16afd88335155c27c34a"}
Sep 06 10:52:06.637928 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:06.637893    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7820-5b6f07"
Sep 06 10:52:06.656400 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:06.656350    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7820-5b6f07_5d715cb1-6e33-46c2-b25d-9219d152d70d/mt-curl-7820-5b6f07/0.log"
Sep 06 10:52:06.656565 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:06.656519    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2x7j\" (UniqueName: \"kubernetes.io/projected/5d715cb1-6e33-46c2-b25d-9219d152d70d-kube-api-access-d2x7j\") pod \"5d715cb1-6e33-46c2-b25d-9219d152d70d\" (UID: \"5d715cb1-6e33-46c2-b25d-9219d152d70d\") "
Sep 06 10:52:06.658797 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:06.658766    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d715cb1-6e33-46c2-b25d-9219d152d70d-kube-api-access-d2x7j" (OuterVolumeSpecName: "kube-api-access-d2x7j") pod "5d715cb1-6e33-46c2-b25d-9219d152d70d" (UID: "5d715cb1-6e33-46c2-b25d-9219d152d70d"). InnerVolumeSpecName "kube-api-access-d2x7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:52:06.681688 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:06.681651    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7820-5b6f07"]
Sep 06 10:52:06.684939 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:06.684915    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7820-5b6f07"]
Sep 06 10:52:06.758267 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:06.758199    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2x7j\" (UniqueName: \"kubernetes.io/projected/5d715cb1-6e33-46c2-b25d-9219d152d70d-kube-api-access-d2x7j\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:07.507326 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:07.507293    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ff941bf876acc35f44e6e2c27f1c156013b6d886676d2a0a7a22d8ecf1cdbb2"
Sep 06 10:52:07.507326 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:07.507322    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7820-5b6f07"
Sep 06 10:52:07.516985 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:52:07.516961    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7820-5b6f07\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="5d715cb1-6e33-46c2-b25d-9219d152d70d" pod="opendatahub/mt-curl-7820-5b6f07"
Sep 06 10:52:08.035239 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:52:08.035203    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7820-5b6f07\" is forbidden: User \"system:node:ip-10-0-133-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-153.ec2.internal' and this object" podUID="5d715cb1-6e33-46c2-b25d-9219d152d70d" pod="opendatahub/mt-curl-7820-5b6f07"
Sep 06 10:52:08.036841 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:08.036819    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d715cb1-6e33-46c2-b25d-9219d152d70d" path="/var/lib/kubelet/pods/5d715cb1-6e33-46c2-b25d-9219d152d70d/volumes"
Sep 06 10:52:11.329549 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:11.329438    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"]
Sep 06 10:52:11.330014 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:11.329677    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj" podUID="7d12595e-1d5f-41b5-8382-0331e41279f5" containerName="maas-api" containerID="cri-o://b777acfaf3258fa0b626b65c1defaea477e6e24faa867fcad420217e0958bdf3" gracePeriod=30
Sep 06 10:52:11.531992 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:11.531931    2557 generic.go:356] "Generic (PLEG): container finished" podID="7d12595e-1d5f-41b5-8382-0331e41279f5" containerID="b777acfaf3258fa0b626b65c1defaea477e6e24faa867fcad420217e0958bdf3" exitCode=0
Sep 06 10:52:11.532213 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:11.532061    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj" event={"ID":"7d12595e-1d5f-41b5-8382-0331e41279f5","Type":"ContainerDied","Data":"b777acfaf3258fa0b626b65c1defaea477e6e24faa867fcad420217e0958bdf3"}
Sep 06 10:52:11.573954 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:11.573931    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"
Sep 06 10:52:11.604769 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:11.604745    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d12595e-1d5f-41b5-8382-0331e41279f5-metrics-certs\") pod \"7d12595e-1d5f-41b5-8382-0331e41279f5\" (UID: \"7d12595e-1d5f-41b5-8382-0331e41279f5\") "
Sep 06 10:52:11.604941 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:11.604796    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d12595e-1d5f-41b5-8382-0331e41279f5-maas-api-tls\") pod \"7d12595e-1d5f-41b5-8382-0331e41279f5\" (UID: \"7d12595e-1d5f-41b5-8382-0331e41279f5\") "
Sep 06 10:52:11.604941 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:11.604827    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzz44\" (UniqueName: \"kubernetes.io/projected/7d12595e-1d5f-41b5-8382-0331e41279f5-kube-api-access-qzz44\") pod \"7d12595e-1d5f-41b5-8382-0331e41279f5\" (UID: \"7d12595e-1d5f-41b5-8382-0331e41279f5\") "
Sep 06 10:52:11.604941 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:11.604863    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d12595e-1d5f-41b5-8382-0331e41279f5-openshift-service-ca\") pod \"7d12595e-1d5f-41b5-8382-0331e41279f5\" (UID: \"7d12595e-1d5f-41b5-8382-0331e41279f5\") "
Sep 06 10:52:11.605272 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:11.605242    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d12595e-1d5f-41b5-8382-0331e41279f5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7d12595e-1d5f-41b5-8382-0331e41279f5" (UID: "7d12595e-1d5f-41b5-8382-0331e41279f5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:52:11.607080 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:11.607013    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d12595e-1d5f-41b5-8382-0331e41279f5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7d12595e-1d5f-41b5-8382-0331e41279f5" (UID: "7d12595e-1d5f-41b5-8382-0331e41279f5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 10:52:11.607080 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:11.607037    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d12595e-1d5f-41b5-8382-0331e41279f5-kube-api-access-qzz44" (OuterVolumeSpecName: "kube-api-access-qzz44") pod "7d12595e-1d5f-41b5-8382-0331e41279f5" (UID: "7d12595e-1d5f-41b5-8382-0331e41279f5"). InnerVolumeSpecName "kube-api-access-qzz44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:52:11.607244 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:11.607208    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d12595e-1d5f-41b5-8382-0331e41279f5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7d12595e-1d5f-41b5-8382-0331e41279f5" (UID: "7d12595e-1d5f-41b5-8382-0331e41279f5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 10:52:11.705888 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:11.705862    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d12595e-1d5f-41b5-8382-0331e41279f5-metrics-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:11.705888 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:11.705886    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d12595e-1d5f-41b5-8382-0331e41279f5-maas-api-tls\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:11.705888 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:11.705894    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzz44\" (UniqueName: \"kubernetes.io/projected/7d12595e-1d5f-41b5-8382-0331e41279f5-kube-api-access-qzz44\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:11.706099 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:11.705904    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d12595e-1d5f-41b5-8382-0331e41279f5-openshift-service-ca\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:12.539039 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:12.539006    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"
Sep 06 10:52:12.539519 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:12.539007    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj" event={"ID":"7d12595e-1d5f-41b5-8382-0331e41279f5","Type":"ContainerDied","Data":"87bef79ccae7a7bebbe114fb58993de793cc7feb06c8e76c92460dd99137de69"}
Sep 06 10:52:12.539519 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:12.539165    2557 scope.go:117] "RemoveContainer" containerID="b777acfaf3258fa0b626b65c1defaea477e6e24faa867fcad420217e0958bdf3"
Sep 06 10:52:12.555617 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:12.555590    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"]
Sep 06 10:52:12.557935 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:12.557913    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-866a8b-76b87dcdc9-kdqpj"]
Sep 06 10:52:14.036762 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:14.036728    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d12595e-1d5f-41b5-8382-0331e41279f5" path="/var/lib/kubelet/pods/7d12595e-1d5f-41b5-8382-0331e41279f5/volumes"
Sep 06 10:52:18.017662 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:18.017626    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"]
Sep 06 10:52:18.018140 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:18.017937    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv" podUID="dceb8afc-b5fe-41ea-822e-d60a1524c7b8" containerName="istio-proxy" containerID="cri-o://5fe80bf7097a28f24cc43260a270f878939f3ef93e83a3fb79570c5bf6fea67e" gracePeriod=30
Sep 06 10:52:22.413873 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:22.413838    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:52:23.313680 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.313653    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:52:23.416316 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.416287    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-credential-socket\") pod \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") "
Sep 06 10:52:23.416316 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.416320    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-token\") pod \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") "
Sep 06 10:52:23.416831 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.416346    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-workload-socket\") pod \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") "
Sep 06 10:52:23.416831 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.416394    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-podinfo\") pod \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") "
Sep 06 10:52:23.416831 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.416437    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wftth\" (UniqueName: \"kubernetes.io/projected/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-kube-api-access-wftth\") pod \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") "
Sep 06 10:52:23.416831 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.416477    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-data\") pod \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") "
Sep 06 10:52:23.416831 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.416563    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istiod-ca-cert\") pod \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") "
Sep 06 10:52:23.416831 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.416592    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dceb8afc-b5fe-41ea-822e-d60a1524c7b8" (UID: "dceb8afc-b5fe-41ea-822e-d60a1524c7b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:52:23.416831 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.416604    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-workload-certs\") pod \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") "
Sep 06 10:52:23.416831 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.416690    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-envoy\") pod \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\" (UID: \"dceb8afc-b5fe-41ea-822e-d60a1524c7b8\") "
Sep 06 10:52:23.416831 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.416773    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dceb8afc-b5fe-41ea-822e-d60a1524c7b8" (UID: "dceb8afc-b5fe-41ea-822e-d60a1524c7b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:52:23.417323 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.416886    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dceb8afc-b5fe-41ea-822e-d60a1524c7b8" (UID: "dceb8afc-b5fe-41ea-822e-d60a1524c7b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:52:23.417323 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.416900    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-data" (OuterVolumeSpecName: "istio-data") pod "dceb8afc-b5fe-41ea-822e-d60a1524c7b8" (UID: "dceb8afc-b5fe-41ea-822e-d60a1524c7b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:52:23.417323 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.417135    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-workload-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:23.417323 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.417156    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-credential-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:23.417323 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.417171    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-workload-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:23.417323 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.417186    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-data\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:23.417323 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.417170    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dceb8afc-b5fe-41ea-822e-d60a1524c7b8" (UID: "dceb8afc-b5fe-41ea-822e-d60a1524c7b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:52:23.418655 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.418623    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "dceb8afc-b5fe-41ea-822e-d60a1524c7b8" (UID: "dceb8afc-b5fe-41ea-822e-d60a1524c7b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:52:23.418795 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.418712    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-kube-api-access-wftth" (OuterVolumeSpecName: "kube-api-access-wftth") pod "dceb8afc-b5fe-41ea-822e-d60a1524c7b8" (UID: "dceb8afc-b5fe-41ea-822e-d60a1524c7b8"). InnerVolumeSpecName "kube-api-access-wftth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:52:23.418979 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.418960    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dceb8afc-b5fe-41ea-822e-d60a1524c7b8" (UID: "dceb8afc-b5fe-41ea-822e-d60a1524c7b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 10:52:23.419045 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.419001    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dceb8afc-b5fe-41ea-822e-d60a1524c7b8" (UID: "dceb8afc-b5fe-41ea-822e-d60a1524c7b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:52:23.517843 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.517799    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istiod-ca-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:23.517843 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.517837    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-envoy\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:23.517843 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.517850    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:23.518063 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.517861    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-istio-podinfo\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:23.518063 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.517873    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wftth\" (UniqueName: \"kubernetes.io/projected/dceb8afc-b5fe-41ea-822e-d60a1524c7b8-kube-api-access-wftth\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:23.602441 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.602406    2557 generic.go:356] "Generic (PLEG): container finished" podID="dceb8afc-b5fe-41ea-822e-d60a1524c7b8" containerID="5fe80bf7097a28f24cc43260a270f878939f3ef93e83a3fb79570c5bf6fea67e" exitCode=0
Sep 06 10:52:23.602614 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.602478    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv" event={"ID":"dceb8afc-b5fe-41ea-822e-d60a1524c7b8","Type":"ContainerDied","Data":"5fe80bf7097a28f24cc43260a270f878939f3ef93e83a3fb79570c5bf6fea67e"}
Sep 06 10:52:23.602614 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.602573    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv" event={"ID":"dceb8afc-b5fe-41ea-822e-d60a1524c7b8","Type":"ContainerDied","Data":"fc5aa256130ef92079920d94c10e21a63df9dec469aec12bfa71691c1eb96572"}
Sep 06 10:52:23.602614 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.602481    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"
Sep 06 10:52:23.602728 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.602596    2557 scope.go:117] "RemoveContainer" containerID="5fe80bf7097a28f24cc43260a270f878939f3ef93e83a3fb79570c5bf6fea67e"
Sep 06 10:52:23.612289 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.612272    2557 scope.go:117] "RemoveContainer" containerID="b6837a8f7acb2c8e17751c022ac21bcb6f745ef85c3bb9b28335c50dc73e13ab"
Sep 06 10:52:23.620765 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.620748    2557 scope.go:117] "RemoveContainer" containerID="5fe80bf7097a28f24cc43260a270f878939f3ef93e83a3fb79570c5bf6fea67e"
Sep 06 10:52:23.621084 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:52:23.621061    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fe80bf7097a28f24cc43260a270f878939f3ef93e83a3fb79570c5bf6fea67e\": container with ID starting with 5fe80bf7097a28f24cc43260a270f878939f3ef93e83a3fb79570c5bf6fea67e not found: ID does not exist" containerID="5fe80bf7097a28f24cc43260a270f878939f3ef93e83a3fb79570c5bf6fea67e"
Sep 06 10:52:23.621168 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.621091    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fe80bf7097a28f24cc43260a270f878939f3ef93e83a3fb79570c5bf6fea67e"} err="failed to get container status \"5fe80bf7097a28f24cc43260a270f878939f3ef93e83a3fb79570c5bf6fea67e\": rpc error: code = NotFound desc = could not find container \"5fe80bf7097a28f24cc43260a270f878939f3ef93e83a3fb79570c5bf6fea67e\": container with ID starting with 5fe80bf7097a28f24cc43260a270f878939f3ef93e83a3fb79570c5bf6fea67e not found: ID does not exist"
Sep 06 10:52:23.621168 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.621108    2557 scope.go:117] "RemoveContainer" containerID="b6837a8f7acb2c8e17751c022ac21bcb6f745ef85c3bb9b28335c50dc73e13ab"
Sep 06 10:52:23.621422 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:52:23.621391    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6837a8f7acb2c8e17751c022ac21bcb6f745ef85c3bb9b28335c50dc73e13ab\": container with ID starting with b6837a8f7acb2c8e17751c022ac21bcb6f745ef85c3bb9b28335c50dc73e13ab not found: ID does not exist" containerID="b6837a8f7acb2c8e17751c022ac21bcb6f745ef85c3bb9b28335c50dc73e13ab"
Sep 06 10:52:23.621579 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.621432    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6837a8f7acb2c8e17751c022ac21bcb6f745ef85c3bb9b28335c50dc73e13ab"} err="failed to get container status \"b6837a8f7acb2c8e17751c022ac21bcb6f745ef85c3bb9b28335c50dc73e13ab\": rpc error: code = NotFound desc = could not find container \"b6837a8f7acb2c8e17751c022ac21bcb6f745ef85c3bb9b28335c50dc73e13ab\": container with ID starting with b6837a8f7acb2c8e17751c022ac21bcb6f745ef85c3bb9b28335c50dc73e13ab not found: ID does not exist"
Sep 06 10:52:23.624125 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.624104    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"]
Sep 06 10:52:23.627263 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:23.627242    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-866a8b-openshift-default-585855d566-lpffv"]
Sep 06 10:52:24.037592 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:24.037519    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dceb8afc-b5fe-41ea-822e-d60a1524c7b8" path="/var/lib/kubelet/pods/dceb8afc-b5fe-41ea-822e-d60a1524c7b8/volumes"
Sep 06 10:52:33.938175 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:33.938143    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"]
Sep 06 10:52:33.938615 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:33.938370    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs" podUID="845f246a-3957-4dde-8956-85a03dc8ecea" containerName="maas-api" containerID="cri-o://b5e255358dc3d4edd8a31ec2bac24dc3060b4eb89627a68bbd7f394739db71a0" gracePeriod=30
Sep 06 10:52:34.181948 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.181923    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:52:34.316320 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.316221    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/845f246a-3957-4dde-8956-85a03dc8ecea-maas-api-tls\") pod \"845f246a-3957-4dde-8956-85a03dc8ecea\" (UID: \"845f246a-3957-4dde-8956-85a03dc8ecea\") "
Sep 06 10:52:34.316320 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.316278    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845f246a-3957-4dde-8956-85a03dc8ecea-metrics-certs\") pod \"845f246a-3957-4dde-8956-85a03dc8ecea\" (UID: \"845f246a-3957-4dde-8956-85a03dc8ecea\") "
Sep 06 10:52:34.316589 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.316378    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/845f246a-3957-4dde-8956-85a03dc8ecea-openshift-service-ca\") pod \"845f246a-3957-4dde-8956-85a03dc8ecea\" (UID: \"845f246a-3957-4dde-8956-85a03dc8ecea\") "
Sep 06 10:52:34.316589 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.316426    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9629h\" (UniqueName: \"kubernetes.io/projected/845f246a-3957-4dde-8956-85a03dc8ecea-kube-api-access-9629h\") pod \"845f246a-3957-4dde-8956-85a03dc8ecea\" (UID: \"845f246a-3957-4dde-8956-85a03dc8ecea\") "
Sep 06 10:52:34.317115 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.317067    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/845f246a-3957-4dde-8956-85a03dc8ecea-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "845f246a-3957-4dde-8956-85a03dc8ecea" (UID: "845f246a-3957-4dde-8956-85a03dc8ecea"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:52:34.319471 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.319436    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/845f246a-3957-4dde-8956-85a03dc8ecea-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "845f246a-3957-4dde-8956-85a03dc8ecea" (UID: "845f246a-3957-4dde-8956-85a03dc8ecea"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 10:52:34.321018 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.320097    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/845f246a-3957-4dde-8956-85a03dc8ecea-kube-api-access-9629h" (OuterVolumeSpecName: "kube-api-access-9629h") pod "845f246a-3957-4dde-8956-85a03dc8ecea" (UID: "845f246a-3957-4dde-8956-85a03dc8ecea"). InnerVolumeSpecName "kube-api-access-9629h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:52:34.322525 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.322473    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/845f246a-3957-4dde-8956-85a03dc8ecea-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "845f246a-3957-4dde-8956-85a03dc8ecea" (UID: "845f246a-3957-4dde-8956-85a03dc8ecea"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 10:52:34.418312 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.418277    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9629h\" (UniqueName: \"kubernetes.io/projected/845f246a-3957-4dde-8956-85a03dc8ecea-kube-api-access-9629h\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:34.418312 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.418306    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/845f246a-3957-4dde-8956-85a03dc8ecea-maas-api-tls\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:34.418312 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.418322    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845f246a-3957-4dde-8956-85a03dc8ecea-metrics-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:34.418580 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.418332    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/845f246a-3957-4dde-8956-85a03dc8ecea-openshift-service-ca\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:34.672885 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.672852    2557 generic.go:356] "Generic (PLEG): container finished" podID="845f246a-3957-4dde-8956-85a03dc8ecea" containerID="b5e255358dc3d4edd8a31ec2bac24dc3060b4eb89627a68bbd7f394739db71a0" exitCode=0
Sep 06 10:52:34.673092 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.672931    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"
Sep 06 10:52:34.673092 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.672946    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs" event={"ID":"845f246a-3957-4dde-8956-85a03dc8ecea","Type":"ContainerDied","Data":"b5e255358dc3d4edd8a31ec2bac24dc3060b4eb89627a68bbd7f394739db71a0"}
Sep 06 10:52:34.673092 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.673026    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs" event={"ID":"845f246a-3957-4dde-8956-85a03dc8ecea","Type":"ContainerDied","Data":"63f10568a70a784b2c3a77b6a11aaacf82667b19478f760f3b52ae4a999dc7a3"}
Sep 06 10:52:34.673092 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.673049    2557 scope.go:117] "RemoveContainer" containerID="b5e255358dc3d4edd8a31ec2bac24dc3060b4eb89627a68bbd7f394739db71a0"
Sep 06 10:52:34.683709 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.683694    2557 scope.go:117] "RemoveContainer" containerID="b5e255358dc3d4edd8a31ec2bac24dc3060b4eb89627a68bbd7f394739db71a0"
Sep 06 10:52:34.683951 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:52:34.683934    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5e255358dc3d4edd8a31ec2bac24dc3060b4eb89627a68bbd7f394739db71a0\": container with ID starting with b5e255358dc3d4edd8a31ec2bac24dc3060b4eb89627a68bbd7f394739db71a0 not found: ID does not exist" containerID="b5e255358dc3d4edd8a31ec2bac24dc3060b4eb89627a68bbd7f394739db71a0"
Sep 06 10:52:34.683991 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.683960    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5e255358dc3d4edd8a31ec2bac24dc3060b4eb89627a68bbd7f394739db71a0"} err="failed to get container status \"b5e255358dc3d4edd8a31ec2bac24dc3060b4eb89627a68bbd7f394739db71a0\": rpc error: code = NotFound desc = could not find container \"b5e255358dc3d4edd8a31ec2bac24dc3060b4eb89627a68bbd7f394739db71a0\": container with ID starting with b5e255358dc3d4edd8a31ec2bac24dc3060b4eb89627a68bbd7f394739db71a0 not found: ID does not exist"
Sep 06 10:52:34.695235 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.695214    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"]
Sep 06 10:52:34.700458 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:34.700436    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b83cfe-5cc4cf898b-cm6gs"]
Sep 06 10:52:36.037864 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:36.037830    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="845f246a-3957-4dde-8956-85a03dc8ecea" path="/var/lib/kubelet/pods/845f246a-3957-4dde-8956-85a03dc8ecea/volumes"
Sep 06 10:52:40.419518 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:40.419456    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"]
Sep 06 10:52:40.419977 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:40.419742    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4" podUID="c3bebebf-c413-4169-b1e2-68fa22ca8ac7" containerName="istio-proxy" containerID="cri-o://ac34c6755911550b2231da9251344fe9ccefbfd504626c17ebd38614ac1c15f6" gracePeriod=30
Sep 06 10:52:40.714542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:40.714446    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:52:41.043185 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:41.043099    2557 patch_prober.go:28] interesting pod/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 06 10:52:41.043340 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:41.043180    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4" podUID="c3bebebf-c413-4169-b1e2-68fa22ca8ac7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 10:52:45.675621 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.675594    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:52:45.741580 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.741480    2557 generic.go:356] "Generic (PLEG): container finished" podID="c3bebebf-c413-4169-b1e2-68fa22ca8ac7" containerID="ac34c6755911550b2231da9251344fe9ccefbfd504626c17ebd38614ac1c15f6" exitCode=0
Sep 06 10:52:45.741580 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.741521    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4" event={"ID":"c3bebebf-c413-4169-b1e2-68fa22ca8ac7","Type":"ContainerDied","Data":"ac34c6755911550b2231da9251344fe9ccefbfd504626c17ebd38614ac1c15f6"}
Sep 06 10:52:45.741580 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.741546    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"
Sep 06 10:52:45.741791 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.741609    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4" event={"ID":"c3bebebf-c413-4169-b1e2-68fa22ca8ac7","Type":"ContainerDied","Data":"75bebff4eb6f9c38a3eded63e81659c1ae1475cb6a2351aeb29721be146402db"}
Sep 06 10:52:45.741791 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.741634    2557 scope.go:117] "RemoveContainer" containerID="ac34c6755911550b2231da9251344fe9ccefbfd504626c17ebd38614ac1c15f6"
Sep 06 10:52:45.750665 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.750618    2557 scope.go:117] "RemoveContainer" containerID="6ff436e6867bca8d9b43e0cf7091478c915c25accdb1ed092318e9e6623ceddc"
Sep 06 10:52:45.762533 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.762516    2557 scope.go:117] "RemoveContainer" containerID="ac34c6755911550b2231da9251344fe9ccefbfd504626c17ebd38614ac1c15f6"
Sep 06 10:52:45.762855 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:52:45.762835    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac34c6755911550b2231da9251344fe9ccefbfd504626c17ebd38614ac1c15f6\": container with ID starting with ac34c6755911550b2231da9251344fe9ccefbfd504626c17ebd38614ac1c15f6 not found: ID does not exist" containerID="ac34c6755911550b2231da9251344fe9ccefbfd504626c17ebd38614ac1c15f6"
Sep 06 10:52:45.762915 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.762864    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac34c6755911550b2231da9251344fe9ccefbfd504626c17ebd38614ac1c15f6"} err="failed to get container status \"ac34c6755911550b2231da9251344fe9ccefbfd504626c17ebd38614ac1c15f6\": rpc error: code = NotFound desc = could not find container \"ac34c6755911550b2231da9251344fe9ccefbfd504626c17ebd38614ac1c15f6\": container with ID starting with ac34c6755911550b2231da9251344fe9ccefbfd504626c17ebd38614ac1c15f6 not found: ID does not exist"
Sep 06 10:52:45.762915 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.762888    2557 scope.go:117] "RemoveContainer" containerID="6ff436e6867bca8d9b43e0cf7091478c915c25accdb1ed092318e9e6623ceddc"
Sep 06 10:52:45.763156 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:52:45.763140    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ff436e6867bca8d9b43e0cf7091478c915c25accdb1ed092318e9e6623ceddc\": container with ID starting with 6ff436e6867bca8d9b43e0cf7091478c915c25accdb1ed092318e9e6623ceddc not found: ID does not exist" containerID="6ff436e6867bca8d9b43e0cf7091478c915c25accdb1ed092318e9e6623ceddc"
Sep 06 10:52:45.763206 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.763163    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ff436e6867bca8d9b43e0cf7091478c915c25accdb1ed092318e9e6623ceddc"} err="failed to get container status \"6ff436e6867bca8d9b43e0cf7091478c915c25accdb1ed092318e9e6623ceddc\": rpc error: code = NotFound desc = could not find container \"6ff436e6867bca8d9b43e0cf7091478c915c25accdb1ed092318e9e6623ceddc\": container with ID starting with 6ff436e6867bca8d9b43e0cf7091478c915c25accdb1ed092318e9e6623ceddc not found: ID does not exist"
Sep 06 10:52:45.824446 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.824418    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-token\") pod \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") "
Sep 06 10:52:45.824583 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.824459    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-workload-socket\") pod \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") "
Sep 06 10:52:45.824583 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.824525    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istiod-ca-cert\") pod \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") "
Sep 06 10:52:45.824730 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.824586    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-data\") pod \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") "
Sep 06 10:52:45.824730 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.824613    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-credential-socket\") pod \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") "
Sep 06 10:52:45.824730 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.824659    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-workload-certs\") pod \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") "
Sep 06 10:52:45.824730 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.824706    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-podinfo\") pod \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") "
Sep 06 10:52:45.824938 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.824744    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-envoy\") pod \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") "
Sep 06 10:52:45.824938 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.824762    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c3bebebf-c413-4169-b1e2-68fa22ca8ac7" (UID: "c3bebebf-c413-4169-b1e2-68fa22ca8ac7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:52:45.824938 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.824774    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfm9f\" (UniqueName: \"kubernetes.io/projected/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-kube-api-access-nfm9f\") pod \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\" (UID: \"c3bebebf-c413-4169-b1e2-68fa22ca8ac7\") "
Sep 06 10:52:45.824938 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.824891    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-data" (OuterVolumeSpecName: "istio-data") pod "c3bebebf-c413-4169-b1e2-68fa22ca8ac7" (UID: "c3bebebf-c413-4169-b1e2-68fa22ca8ac7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:52:45.824938 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.824880    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c3bebebf-c413-4169-b1e2-68fa22ca8ac7" (UID: "c3bebebf-c413-4169-b1e2-68fa22ca8ac7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:52:45.824938 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.824923    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c3bebebf-c413-4169-b1e2-68fa22ca8ac7" (UID: "c3bebebf-c413-4169-b1e2-68fa22ca8ac7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:52:45.825244 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.824966    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c3bebebf-c413-4169-b1e2-68fa22ca8ac7" (UID: "c3bebebf-c413-4169-b1e2-68fa22ca8ac7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:52:45.825244 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.825186    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-credential-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:45.825244 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.825209    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-workload-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:45.825244 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.825226    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-workload-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:45.825244 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.825243    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istiod-ca-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:45.825531 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.825258    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-data\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:45.826649 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.826623    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-token" (OuterVolumeSpecName: "istio-token") pod "c3bebebf-c413-4169-b1e2-68fa22ca8ac7" (UID: "c3bebebf-c413-4169-b1e2-68fa22ca8ac7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:52:45.826806 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.826787    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-kube-api-access-nfm9f" (OuterVolumeSpecName: "kube-api-access-nfm9f") pod "c3bebebf-c413-4169-b1e2-68fa22ca8ac7" (UID: "c3bebebf-c413-4169-b1e2-68fa22ca8ac7"). InnerVolumeSpecName "kube-api-access-nfm9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:52:45.826881 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.826862    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c3bebebf-c413-4169-b1e2-68fa22ca8ac7" (UID: "c3bebebf-c413-4169-b1e2-68fa22ca8ac7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 10:52:45.826991 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.826977    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c3bebebf-c413-4169-b1e2-68fa22ca8ac7" (UID: "c3bebebf-c413-4169-b1e2-68fa22ca8ac7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:52:45.926565 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.926534    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:45.926565 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.926560    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-podinfo\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:45.926565 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.926570    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-istio-envoy\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:45.926764 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:45.926578    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfm9f\" (UniqueName: \"kubernetes.io/projected/c3bebebf-c413-4169-b1e2-68fa22ca8ac7-kube-api-access-nfm9f\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:52:46.063963 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:46.063932    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"]
Sep 06 10:52:46.066934 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:46.066900    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-b83cfe-openshift-default-7c494b587b-qskn4"]
Sep 06 10:52:48.037779 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:48.037747    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3bebebf-c413-4169-b1e2-68fa22ca8ac7" path="/var/lib/kubelet/pods/c3bebebf-c413-4169-b1e2-68fa22ca8ac7/volumes"
Sep 06 10:52:51.521332 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:51.521294    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:52:57.777440 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:57.777405    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"]
Sep 06 10:52:57.777990 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:57.777653    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5" podUID="475cab92-806a-4de2-bc9b-2b33485eba74" containerName="istio-proxy" containerID="cri-o://101e19f5911f1e5335d4e598b4914473a510b34888d02e764a0c291f46b25a42" gracePeriod=30
Sep 06 10:52:58.116707 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.116673    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-fbf820-g87bv"]
Sep 06 10:52:58.117053 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.117041    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="845f246a-3957-4dde-8956-85a03dc8ecea" containerName="maas-api"
Sep 06 10:52:58.117105 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.117055    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="845f246a-3957-4dde-8956-85a03dc8ecea" containerName="maas-api"
Sep 06 10:52:58.117105 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.117065    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d715cb1-6e33-46c2-b25d-9219d152d70d" containerName="mt-curl-7820-5b6f07"
Sep 06 10:52:58.117105 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.117072    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d715cb1-6e33-46c2-b25d-9219d152d70d" containerName="mt-curl-7820-5b6f07"
Sep 06 10:52:58.117105 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.117089    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3bebebf-c413-4169-b1e2-68fa22ca8ac7" containerName="istio-proxy"
Sep 06 10:52:58.117105 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.117094    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3bebebf-c413-4169-b1e2-68fa22ca8ac7" containerName="istio-proxy"
Sep 06 10:52:58.117105 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.117102    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3bebebf-c413-4169-b1e2-68fa22ca8ac7" containerName="istio-proxy"
Sep 06 10:52:58.117105 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.117107    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3bebebf-c413-4169-b1e2-68fa22ca8ac7" containerName="istio-proxy"
Sep 06 10:52:58.117320 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.117116    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d12595e-1d5f-41b5-8382-0331e41279f5" containerName="maas-api"
Sep 06 10:52:58.117320 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.117121    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d12595e-1d5f-41b5-8382-0331e41279f5" containerName="maas-api"
Sep 06 10:52:58.117320 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.117127    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dceb8afc-b5fe-41ea-822e-d60a1524c7b8" containerName="istio-proxy"
Sep 06 10:52:58.117320 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.117133    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dceb8afc-b5fe-41ea-822e-d60a1524c7b8" containerName="istio-proxy"
Sep 06 10:52:58.117320 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.117193    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3bebebf-c413-4169-b1e2-68fa22ca8ac7" containerName="istio-proxy"
Sep 06 10:52:58.117320 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.117199    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d715cb1-6e33-46c2-b25d-9219d152d70d" containerName="mt-curl-7820-5b6f07"
Sep 06 10:52:58.117320 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.117205    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="dceb8afc-b5fe-41ea-822e-d60a1524c7b8" containerName="istio-proxy"
Sep 06 10:52:58.117320 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.117213    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d12595e-1d5f-41b5-8382-0331e41279f5" containerName="maas-api"
Sep 06 10:52:58.117320 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.117220    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="845f246a-3957-4dde-8956-85a03dc8ecea" containerName="maas-api"
Sep 06 10:52:58.121870 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.121849    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-fbf820-g87bv"
Sep 06 10:52:58.124741 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.124705    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 10:52:58.124920 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.124766    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\""
Sep 06 10:52:58.126422 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.126395    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-fbf820-g87bv"]
Sep 06 10:52:58.136792 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.136768    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-fbf820-g87bv\" (UID: \"5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-fbf820-g87bv"
Sep 06 10:52:58.237566 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.237536    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-fbf820-g87bv\" (UID: \"5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-fbf820-g87bv"
Sep 06 10:52:58.238142 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.238120    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-fbf820-g87bv\" (UID: \"5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-fbf820-g87bv"
Sep 06 10:52:58.434023 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.433932    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-fbf820-g87bv"
Sep 06 10:52:58.712335 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.712252    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-fbf820-g87bv"]
Sep 06 10:52:58.715812 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:52:58.715788    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ad7b8c5_b958_4ff8_99df_7b715d1dfe1f.slice/crio-bce61c9ab1b438220073666037da2e0af0dbe05e9d4c51e5807de1fc2e0dc436 WatchSource:0}: Error finding container bce61c9ab1b438220073666037da2e0af0dbe05e9d4c51e5807de1fc2e0dc436: Status 404 returned error can't find the container with id bce61c9ab1b438220073666037da2e0af0dbe05e9d4c51e5807de1fc2e0dc436
Sep 06 10:52:58.811689 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:58.811662    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-fbf820-g87bv" event={"ID":"5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f","Type":"ContainerStarted","Data":"bce61c9ab1b438220073666037da2e0af0dbe05e9d4c51e5807de1fc2e0dc436"}
Sep 06 10:52:59.818442 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:59.818359    2557 generic.go:356] "Generic (PLEG): container finished" podID="5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f" containerID="676fecd529c19d08e39c36f80053909ae799dc0b8d3fd4c3c7dd597595c9aa1a" exitCode=0
Sep 06 10:52:59.818819 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:52:59.818451    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-fbf820-g87bv" event={"ID":"5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f","Type":"ContainerDied","Data":"676fecd529c19d08e39c36f80053909ae799dc0b8d3fd4c3c7dd597595c9aa1a"}
Sep 06 10:53:00.958142 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:00.958119    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-fbf820-g87bv"
Sep 06 10:53:01.062830 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:01.062787    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f-maas-api-service-ca\") pod \"5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f\" (UID: \"5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f\") "
Sep 06 10:53:01.063169 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:01.063146    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f" (UID: "5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:53:01.164426 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:01.164396    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f-maas-api-service-ca\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:53:01.830553 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:01.830524    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-fbf820-g87bv"
Sep 06 10:53:01.830718 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:01.830528    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-fbf820-g87bv" event={"ID":"5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f","Type":"ContainerDied","Data":"bce61c9ab1b438220073666037da2e0af0dbe05e9d4c51e5807de1fc2e0dc436"}
Sep 06 10:53:01.830718 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:01.830642    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bce61c9ab1b438220073666037da2e0af0dbe05e9d4c51e5807de1fc2e0dc436"
Sep 06 10:53:02.910761 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:02.910731    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:53:03.029460 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.029435    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:53:03.081167 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.081099    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/475cab92-806a-4de2-bc9b-2b33485eba74-istiod-ca-cert\") pod \"475cab92-806a-4de2-bc9b-2b33485eba74\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") "
Sep 06 10:53:03.081167 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.081163    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-istio-envoy\") pod \"475cab92-806a-4de2-bc9b-2b33485eba74\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") "
Sep 06 10:53:03.081352 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.081333    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-credential-socket\") pod \"475cab92-806a-4de2-bc9b-2b33485eba74\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") "
Sep 06 10:53:03.081396 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.081378    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/475cab92-806a-4de2-bc9b-2b33485eba74-istio-podinfo\") pod \"475cab92-806a-4de2-bc9b-2b33485eba74\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") "
Sep 06 10:53:03.081431 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.081415    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-workload-socket\") pod \"475cab92-806a-4de2-bc9b-2b33485eba74\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") "
Sep 06 10:53:03.081463 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.081442    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/475cab92-806a-4de2-bc9b-2b33485eba74-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "475cab92-806a-4de2-bc9b-2b33485eba74" (UID: "475cab92-806a-4de2-bc9b-2b33485eba74"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:53:03.081530 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.081465    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8x7sl\" (UniqueName: \"kubernetes.io/projected/475cab92-806a-4de2-bc9b-2b33485eba74-kube-api-access-8x7sl\") pod \"475cab92-806a-4de2-bc9b-2b33485eba74\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") "
Sep 06 10:53:03.081582 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.081532    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/475cab92-806a-4de2-bc9b-2b33485eba74-istio-token\") pod \"475cab92-806a-4de2-bc9b-2b33485eba74\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") "
Sep 06 10:53:03.081582 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.081559    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-istio-data\") pod \"475cab92-806a-4de2-bc9b-2b33485eba74\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") "
Sep 06 10:53:03.081684 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.081585    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-workload-certs\") pod \"475cab92-806a-4de2-bc9b-2b33485eba74\" (UID: \"475cab92-806a-4de2-bc9b-2b33485eba74\") "
Sep 06 10:53:03.081815 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.081582    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "475cab92-806a-4de2-bc9b-2b33485eba74" (UID: "475cab92-806a-4de2-bc9b-2b33485eba74"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:53:03.081815 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.081687    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "475cab92-806a-4de2-bc9b-2b33485eba74" (UID: "475cab92-806a-4de2-bc9b-2b33485eba74"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:53:03.082019 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.081837    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-istio-data" (OuterVolumeSpecName: "istio-data") pod "475cab92-806a-4de2-bc9b-2b33485eba74" (UID: "475cab92-806a-4de2-bc9b-2b33485eba74"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:53:03.082019 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.081850    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "475cab92-806a-4de2-bc9b-2b33485eba74" (UID: "475cab92-806a-4de2-bc9b-2b33485eba74"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:53:03.082019 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.082001    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-istio-data\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:53:03.082192 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.082023    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-workload-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:53:03.082192 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.082042    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/475cab92-806a-4de2-bc9b-2b33485eba74-istiod-ca-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:53:03.082192 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.082057    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-credential-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:53:03.082192 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.082071    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-workload-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:53:03.083393 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.083366    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "475cab92-806a-4de2-bc9b-2b33485eba74" (UID: "475cab92-806a-4de2-bc9b-2b33485eba74"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:53:03.083635 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.083606    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/475cab92-806a-4de2-bc9b-2b33485eba74-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "475cab92-806a-4de2-bc9b-2b33485eba74" (UID: "475cab92-806a-4de2-bc9b-2b33485eba74"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 10:53:03.083711 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.083687    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/475cab92-806a-4de2-bc9b-2b33485eba74-istio-token" (OuterVolumeSpecName: "istio-token") pod "475cab92-806a-4de2-bc9b-2b33485eba74" (UID: "475cab92-806a-4de2-bc9b-2b33485eba74"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:53:03.083765 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.083742    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/475cab92-806a-4de2-bc9b-2b33485eba74-kube-api-access-8x7sl" (OuterVolumeSpecName: "kube-api-access-8x7sl") pod "475cab92-806a-4de2-bc9b-2b33485eba74" (UID: "475cab92-806a-4de2-bc9b-2b33485eba74"). InnerVolumeSpecName "kube-api-access-8x7sl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:53:03.182676 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.182643    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/475cab92-806a-4de2-bc9b-2b33485eba74-istio-envoy\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:53:03.182676 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.182670    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/475cab92-806a-4de2-bc9b-2b33485eba74-istio-podinfo\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:53:03.182676 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.182680    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8x7sl\" (UniqueName: \"kubernetes.io/projected/475cab92-806a-4de2-bc9b-2b33485eba74-kube-api-access-8x7sl\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:53:03.182897 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.182690    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/475cab92-806a-4de2-bc9b-2b33485eba74-istio-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:53:03.843336 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.843296    2557 generic.go:356] "Generic (PLEG): container finished" podID="475cab92-806a-4de2-bc9b-2b33485eba74" containerID="101e19f5911f1e5335d4e598b4914473a510b34888d02e764a0c291f46b25a42" exitCode=0
Sep 06 10:53:03.843553 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.843344    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"
Sep 06 10:53:03.843553 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.843383    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5" event={"ID":"475cab92-806a-4de2-bc9b-2b33485eba74","Type":"ContainerDied","Data":"101e19f5911f1e5335d4e598b4914473a510b34888d02e764a0c291f46b25a42"}
Sep 06 10:53:03.843553 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.843433    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5" event={"ID":"475cab92-806a-4de2-bc9b-2b33485eba74","Type":"ContainerDied","Data":"88c7acd7f01f6f702096c808b488b63ad67aa22d68b70a0629067c0c9d96775d"}
Sep 06 10:53:03.843553 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.843455    2557 scope.go:117] "RemoveContainer" containerID="101e19f5911f1e5335d4e598b4914473a510b34888d02e764a0c291f46b25a42"
Sep 06 10:53:03.854630 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.854612    2557 scope.go:117] "RemoveContainer" containerID="d21dba38d774e7059070945a5b9ce8bb77a3192122705baa7370c772a572e401"
Sep 06 10:53:03.862896 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.862877    2557 scope.go:117] "RemoveContainer" containerID="101e19f5911f1e5335d4e598b4914473a510b34888d02e764a0c291f46b25a42"
Sep 06 10:53:03.863241 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:53:03.863216    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"101e19f5911f1e5335d4e598b4914473a510b34888d02e764a0c291f46b25a42\": container with ID starting with 101e19f5911f1e5335d4e598b4914473a510b34888d02e764a0c291f46b25a42 not found: ID does not exist" containerID="101e19f5911f1e5335d4e598b4914473a510b34888d02e764a0c291f46b25a42"
Sep 06 10:53:03.863353 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.863250    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"101e19f5911f1e5335d4e598b4914473a510b34888d02e764a0c291f46b25a42"} err="failed to get container status \"101e19f5911f1e5335d4e598b4914473a510b34888d02e764a0c291f46b25a42\": rpc error: code = NotFound desc = could not find container \"101e19f5911f1e5335d4e598b4914473a510b34888d02e764a0c291f46b25a42\": container with ID starting with 101e19f5911f1e5335d4e598b4914473a510b34888d02e764a0c291f46b25a42 not found: ID does not exist"
Sep 06 10:53:03.863353 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.863273    2557 scope.go:117] "RemoveContainer" containerID="d21dba38d774e7059070945a5b9ce8bb77a3192122705baa7370c772a572e401"
Sep 06 10:53:03.863630 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:53:03.863605    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d21dba38d774e7059070945a5b9ce8bb77a3192122705baa7370c772a572e401\": container with ID starting with d21dba38d774e7059070945a5b9ce8bb77a3192122705baa7370c772a572e401 not found: ID does not exist" containerID="d21dba38d774e7059070945a5b9ce8bb77a3192122705baa7370c772a572e401"
Sep 06 10:53:03.863704 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.863638    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d21dba38d774e7059070945a5b9ce8bb77a3192122705baa7370c772a572e401"} err="failed to get container status \"d21dba38d774e7059070945a5b9ce8bb77a3192122705baa7370c772a572e401\": rpc error: code = NotFound desc = could not find container \"d21dba38d774e7059070945a5b9ce8bb77a3192122705baa7370c772a572e401\": container with ID starting with d21dba38d774e7059070945a5b9ce8bb77a3192122705baa7370c772a572e401 not found: ID does not exist"
Sep 06 10:53:03.866038 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.865904    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"]
Sep 06 10:53:03.868831 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:03.868805    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-72cb16-openshift-default-665bcd8d6d-fvvj5"]
Sep 06 10:53:04.037174 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:04.037140    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="475cab92-806a-4de2-bc9b-2b33485eba74" path="/var/lib/kubelet/pods/475cab92-806a-4de2-bc9b-2b33485eba74/volumes"
Sep 06 10:53:08.768685 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:08.768641    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-fbf820-g87bv"]
Sep 06 10:53:08.772834 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:08.772808    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-fbf820-g87bv"]
Sep 06 10:53:10.036967 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:10.036931    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f" path="/var/lib/kubelet/pods/5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f/volumes"
Sep 06 10:53:13.420868 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:13.420833    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:53:15.148192 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:15.148159    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"]
Sep 06 10:53:15.148617 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:15.148361    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff" podUID="3b061474-63c2-4ca7-b309-189cb3bad4d7" containerName="istio-proxy" containerID="cri-o://21da10318eb1d0f27b53a1f1e487fd1844d2e2c94dc0daad4e4b27ad28358e48" gracePeriod=30
Sep 06 10:53:20.158673 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.158635    2557 patch_prober.go:28] interesting pod/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": read tcp 10.132.0.2:33212->10.132.0.66:15021: read: connection reset by peer" start-of-body=
Sep 06 10:53:20.159039 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.158717    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff" podUID="3b061474-63c2-4ca7-b309-189cb3bad4d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": read tcp 10.132.0.2:33212->10.132.0.66:15021: read: connection reset by peer"
Sep 06 10:53:20.405430 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.405407    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:53:20.550668 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.550568    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-workload-certs\") pod \"3b061474-63c2-4ca7-b309-189cb3bad4d7\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") "
Sep 06 10:53:20.550668 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.550606    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-podinfo\") pod \"3b061474-63c2-4ca7-b309-189cb3bad4d7\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") "
Sep 06 10:53:20.550668 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.550644    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-token\") pod \"3b061474-63c2-4ca7-b309-189cb3bad4d7\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") "
Sep 06 10:53:20.550954 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.550763    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b061474-63c2-4ca7-b309-189cb3bad4d7-istiod-ca-cert\") pod \"3b061474-63c2-4ca7-b309-189cb3bad4d7\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") "
Sep 06 10:53:20.550954 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.550795    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-credential-socket\") pod \"3b061474-63c2-4ca7-b309-189cb3bad4d7\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") "
Sep 06 10:53:20.550954 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.550824    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-envoy\") pod \"3b061474-63c2-4ca7-b309-189cb3bad4d7\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") "
Sep 06 10:53:20.550954 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.550870    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-data\") pod \"3b061474-63c2-4ca7-b309-189cb3bad4d7\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") "
Sep 06 10:53:20.550954 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.550897    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rhmv\" (UniqueName: \"kubernetes.io/projected/3b061474-63c2-4ca7-b309-189cb3bad4d7-kube-api-access-5rhmv\") pod \"3b061474-63c2-4ca7-b309-189cb3bad4d7\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") "
Sep 06 10:53:20.551229 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.550970    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-workload-socket\") pod \"3b061474-63c2-4ca7-b309-189cb3bad4d7\" (UID: \"3b061474-63c2-4ca7-b309-189cb3bad4d7\") "
Sep 06 10:53:20.551229 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.550969    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3b061474-63c2-4ca7-b309-189cb3bad4d7" (UID: "3b061474-63c2-4ca7-b309-189cb3bad4d7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:53:20.551229 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.551051    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3b061474-63c2-4ca7-b309-189cb3bad4d7" (UID: "3b061474-63c2-4ca7-b309-189cb3bad4d7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:53:20.551229 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.551125    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-data" (OuterVolumeSpecName: "istio-data") pod "3b061474-63c2-4ca7-b309-189cb3bad4d7" (UID: "3b061474-63c2-4ca7-b309-189cb3bad4d7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:53:20.551229 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.551177    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b061474-63c2-4ca7-b309-189cb3bad4d7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3b061474-63c2-4ca7-b309-189cb3bad4d7" (UID: "3b061474-63c2-4ca7-b309-189cb3bad4d7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:53:20.551542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.551303    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3b061474-63c2-4ca7-b309-189cb3bad4d7" (UID: "3b061474-63c2-4ca7-b309-189cb3bad4d7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:53:20.551542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.551416    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-workload-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:53:20.551542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.551433    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-workload-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:53:20.551542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.551447    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b061474-63c2-4ca7-b309-189cb3bad4d7-istiod-ca-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:53:20.551542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.551458    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-credential-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:53:20.551542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.551470    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-data\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:53:20.552939 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.552907    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3b061474-63c2-4ca7-b309-189cb3bad4d7" (UID: "3b061474-63c2-4ca7-b309-189cb3bad4d7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:53:20.553054 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.552966    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3b061474-63c2-4ca7-b309-189cb3bad4d7" (UID: "3b061474-63c2-4ca7-b309-189cb3bad4d7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 10:53:20.553054 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.553010    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-token" (OuterVolumeSpecName: "istio-token") pod "3b061474-63c2-4ca7-b309-189cb3bad4d7" (UID: "3b061474-63c2-4ca7-b309-189cb3bad4d7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:53:20.553146 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.553076    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b061474-63c2-4ca7-b309-189cb3bad4d7-kube-api-access-5rhmv" (OuterVolumeSpecName: "kube-api-access-5rhmv") pod "3b061474-63c2-4ca7-b309-189cb3bad4d7" (UID: "3b061474-63c2-4ca7-b309-189cb3bad4d7"). InnerVolumeSpecName "kube-api-access-5rhmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:53:20.652948 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.652907    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-podinfo\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:53:20.652948 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.652944    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:53:20.652948 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.652953    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b061474-63c2-4ca7-b309-189cb3bad4d7-istio-envoy\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:53:20.653188 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.652961    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rhmv\" (UniqueName: \"kubernetes.io/projected/3b061474-63c2-4ca7-b309-189cb3bad4d7-kube-api-access-5rhmv\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:53:20.936582 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.936547    2557 generic.go:356] "Generic (PLEG): container finished" podID="3b061474-63c2-4ca7-b309-189cb3bad4d7" containerID="21da10318eb1d0f27b53a1f1e487fd1844d2e2c94dc0daad4e4b27ad28358e48" exitCode=0
Sep 06 10:53:20.936754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.936598    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"
Sep 06 10:53:20.936754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.936629    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff" event={"ID":"3b061474-63c2-4ca7-b309-189cb3bad4d7","Type":"ContainerDied","Data":"21da10318eb1d0f27b53a1f1e487fd1844d2e2c94dc0daad4e4b27ad28358e48"}
Sep 06 10:53:20.936754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.936683    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff" event={"ID":"3b061474-63c2-4ca7-b309-189cb3bad4d7","Type":"ContainerDied","Data":"7518a5df0e6d6d2fb330acfb8b259379a661c4530b368d7662ca97fa134a70ab"}
Sep 06 10:53:20.936754 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.936706    2557 scope.go:117] "RemoveContainer" containerID="21da10318eb1d0f27b53a1f1e487fd1844d2e2c94dc0daad4e4b27ad28358e48"
Sep 06 10:53:20.946272 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.946251    2557 scope.go:117] "RemoveContainer" containerID="dc8a68620f3e1b12c5e3fa266418e280d069bb9de89bf88c93550f5f4d81da2c"
Sep 06 10:53:20.954734 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.954710    2557 scope.go:117] "RemoveContainer" containerID="21da10318eb1d0f27b53a1f1e487fd1844d2e2c94dc0daad4e4b27ad28358e48"
Sep 06 10:53:20.955063 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:53:20.955041    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21da10318eb1d0f27b53a1f1e487fd1844d2e2c94dc0daad4e4b27ad28358e48\": container with ID starting with 21da10318eb1d0f27b53a1f1e487fd1844d2e2c94dc0daad4e4b27ad28358e48 not found: ID does not exist" containerID="21da10318eb1d0f27b53a1f1e487fd1844d2e2c94dc0daad4e4b27ad28358e48"
Sep 06 10:53:20.955180 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.955073    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21da10318eb1d0f27b53a1f1e487fd1844d2e2c94dc0daad4e4b27ad28358e48"} err="failed to get container status \"21da10318eb1d0f27b53a1f1e487fd1844d2e2c94dc0daad4e4b27ad28358e48\": rpc error: code = NotFound desc = could not find container \"21da10318eb1d0f27b53a1f1e487fd1844d2e2c94dc0daad4e4b27ad28358e48\": container with ID starting with 21da10318eb1d0f27b53a1f1e487fd1844d2e2c94dc0daad4e4b27ad28358e48 not found: ID does not exist"
Sep 06 10:53:20.955180 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.955097    2557 scope.go:117] "RemoveContainer" containerID="dc8a68620f3e1b12c5e3fa266418e280d069bb9de89bf88c93550f5f4d81da2c"
Sep 06 10:53:20.955375 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:53:20.955359    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc8a68620f3e1b12c5e3fa266418e280d069bb9de89bf88c93550f5f4d81da2c\": container with ID starting with dc8a68620f3e1b12c5e3fa266418e280d069bb9de89bf88c93550f5f4d81da2c not found: ID does not exist" containerID="dc8a68620f3e1b12c5e3fa266418e280d069bb9de89bf88c93550f5f4d81da2c"
Sep 06 10:53:20.955447 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.955382    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc8a68620f3e1b12c5e3fa266418e280d069bb9de89bf88c93550f5f4d81da2c"} err="failed to get container status \"dc8a68620f3e1b12c5e3fa266418e280d069bb9de89bf88c93550f5f4d81da2c\": rpc error: code = NotFound desc = could not find container \"dc8a68620f3e1b12c5e3fa266418e280d069bb9de89bf88c93550f5f4d81da2c\": container with ID starting with dc8a68620f3e1b12c5e3fa266418e280d069bb9de89bf88c93550f5f4d81da2c not found: ID does not exist"
Sep 06 10:53:20.957798 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.957775    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"]
Sep 06 10:53:20.960740 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:20.960720    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-fbf820-openshift-default-b9dbbc5b-gkvff"]
Sep 06 10:53:22.036928 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:22.036887    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b061474-63c2-4ca7-b309-189cb3bad4d7" path="/var/lib/kubelet/pods/3b061474-63c2-4ca7-b309-189cb3bad4d7/volumes"
Sep 06 10:53:25.512286 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:25.512199    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:53:33.542769 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.542730    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"]
Sep 06 10:53:33.543140 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.543119    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="475cab92-806a-4de2-bc9b-2b33485eba74" containerName="istio-proxy"
Sep 06 10:53:33.543140 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.543130    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="475cab92-806a-4de2-bc9b-2b33485eba74" containerName="istio-proxy"
Sep 06 10:53:33.543140 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.543139    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b061474-63c2-4ca7-b309-189cb3bad4d7" containerName="istio-proxy"
Sep 06 10:53:33.543243 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.543145    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b061474-63c2-4ca7-b309-189cb3bad4d7" containerName="istio-proxy"
Sep 06 10:53:33.543243 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.543155    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="475cab92-806a-4de2-bc9b-2b33485eba74" containerName="istio-proxy"
Sep 06 10:53:33.543243 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.543161    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="475cab92-806a-4de2-bc9b-2b33485eba74" containerName="istio-proxy"
Sep 06 10:53:33.543243 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.543170    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b061474-63c2-4ca7-b309-189cb3bad4d7" containerName="istio-proxy"
Sep 06 10:53:33.543243 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.543176    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b061474-63c2-4ca7-b309-189cb3bad4d7" containerName="istio-proxy"
Sep 06 10:53:33.543243 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.543187    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f" containerName="revoke-keys"
Sep 06 10:53:33.543243 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.543192    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f" containerName="revoke-keys"
Sep 06 10:53:33.543243 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.543209    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dceb8afc-b5fe-41ea-822e-d60a1524c7b8" containerName="istio-proxy"
Sep 06 10:53:33.543243 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.543214    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dceb8afc-b5fe-41ea-822e-d60a1524c7b8" containerName="istio-proxy"
Sep 06 10:53:33.543542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.543274    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="475cab92-806a-4de2-bc9b-2b33485eba74" containerName="istio-proxy"
Sep 06 10:53:33.543542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.543282    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ad7b8c5-b958-4ff8-99df-7b715d1dfe1f" containerName="revoke-keys"
Sep 06 10:53:33.543542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.543289    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b061474-63c2-4ca7-b309-189cb3bad4d7" containerName="istio-proxy"
Sep 06 10:53:33.543542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.543294    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="475cab92-806a-4de2-bc9b-2b33485eba74" containerName="istio-proxy"
Sep 06 10:53:33.543542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.543302    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3bebebf-c413-4169-b1e2-68fa22ca8ac7" containerName="istio-proxy"
Sep 06 10:53:33.543542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.543307    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="dceb8afc-b5fe-41ea-822e-d60a1524c7b8" containerName="istio-proxy"
Sep 06 10:53:33.543542 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.543313    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b061474-63c2-4ca7-b309-189cb3bad4d7" containerName="istio-proxy"
Sep 06 10:53:33.547759 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.547727    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.550046 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.550022    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-69ef21-openshift-default-dockercfg-tcfc5\""
Sep 06 10:53:33.555408 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.555385    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"]
Sep 06 10:53:33.566103 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.566072    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psqdr\" (UniqueName: \"kubernetes.io/projected/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-kube-api-access-psqdr\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.566248 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.566148    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-podinfo\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.566248 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.566193    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-envoy\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.566248 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.566223    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-data\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.566436 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.566289    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-workload-certs\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.566436 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.566358    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-token\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.566436 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.566427    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istiod-ca-cert\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.566614 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.566455    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-credential-socket\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.566614 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.566484    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-workload-socket\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.667423 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.667391    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istiod-ca-cert\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.667423 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.667427    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-credential-socket\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.667713 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.667444    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-workload-socket\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.667713 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.667471    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psqdr\" (UniqueName: \"kubernetes.io/projected/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-kube-api-access-psqdr\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.667713 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.667526    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-podinfo\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.667713 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.667557    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-envoy\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.667713 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.667576    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-data\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.667713 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.667600    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-workload-certs\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.667713 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.667676    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-token\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.668108 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.667922    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-credential-socket\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.668108 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.668051    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-data\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.668213 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.668150    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-workload-socket\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.668287 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.668251    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-workload-certs\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.668478 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.668454    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istiod-ca-cert\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.669961 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.669941    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-envoy\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.670173 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.670154    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-podinfo\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.676369 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.676339    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-token\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.676482 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.676386    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psqdr\" (UniqueName: \"kubernetes.io/projected/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-kube-api-access-psqdr\") pod \"e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") " pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:33.860031 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:33.859999    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:34.134310 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:34.134282    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"]
Sep 06 10:53:34.135891 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:53:34.135856    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dc570f5_e559_48cc_8dcd_95fcae8ee57f.slice/crio-466221e7f47b5cf120bf709b412e7964d79ea021f38ba545dddac865753e83a5 WatchSource:0}: Error finding container 466221e7f47b5cf120bf709b412e7964d79ea021f38ba545dddac865753e83a5: Status 404 returned error can't find the container with id 466221e7f47b5cf120bf709b412e7964d79ea021f38ba545dddac865753e83a5
Sep 06 10:53:34.138126 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:34.138092    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:53:34.138209 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:34.138159    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:53:34.138209 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:34.138185    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:53:34.612049 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:34.612014    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:53:35.016789 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:35.016701    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n" event={"ID":"4dc570f5-e559-48cc-8dcd-95fcae8ee57f","Type":"ContainerStarted","Data":"f229c1ca8ee823546195a6a5a4b11a73897e93b5d9ef21c96f9f685cd262688e"}
Sep 06 10:53:35.016789 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:35.016764    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n" event={"ID":"4dc570f5-e559-48cc-8dcd-95fcae8ee57f","Type":"ContainerStarted","Data":"466221e7f47b5cf120bf709b412e7964d79ea021f38ba545dddac865753e83a5"}
Sep 06 10:53:35.035783 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:35.035737    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n" podStartSLOduration=2.035723452 podStartE2EDuration="2.035723452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:53:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:53:35.034070297 +0000 UTC m=+1427.483851076" watchObservedRunningTime="2026-09-06 10:53:35.035723452 +0000 UTC m=+1427.485504220"
Sep 06 10:53:35.860919 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:35.860883    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:35.866751 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:35.866725    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:36.021788 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:36.021759    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:36.023101 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:36.023083    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:38.130037 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.130004    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"]
Sep 06 10:53:38.133729 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.133711    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"
Sep 06 10:53:38.135963 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.135940    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-69ef21-serving-cert\""
Sep 06 10:53:38.137067 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.137046    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q972h\""
Sep 06 10:53:38.137283 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.137254    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 06 10:53:38.137283 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.137277    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 10:53:38.137545 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.137531    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 06 10:53:38.144667 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.144646    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"]
Sep 06 10:53:38.211773 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.211741    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-metrics-certs\") pod \"maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8\" (UID: \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"
Sep 06 10:53:38.211945 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.211800    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdqzq\" (UniqueName: \"kubernetes.io/projected/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-kube-api-access-jdqzq\") pod \"maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8\" (UID: \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"
Sep 06 10:53:38.211945 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.211858    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8\" (UID: \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"
Sep 06 10:53:38.211945 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.211887    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8\" (UID: \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"
Sep 06 10:53:38.313094 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.313056    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8\" (UID: \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"
Sep 06 10:53:38.313292 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.313118    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-metrics-certs\") pod \"maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8\" (UID: \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"
Sep 06 10:53:38.313292 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.313179    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdqzq\" (UniqueName: \"kubernetes.io/projected/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-kube-api-access-jdqzq\") pod \"maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8\" (UID: \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"
Sep 06 10:53:38.313292 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:53:38.313213    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-serving-cert: secret "maas-api-e2e-ipp-a-69ef21-serving-cert" not found
Sep 06 10:53:38.313292 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.313219    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8\" (UID: \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"
Sep 06 10:53:38.313292 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:53:38.313277    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-maas-api-tls podName:31d11217-4f66-4a6e-99f4-aed0cdcbb87f nodeName:}" failed. No retries permitted until 2026-09-06 10:53:38.813256376 +0000 UTC m=+1431.263037145 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-maas-api-tls") pod "maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8" (UID: "31d11217-4f66-4a6e-99f4-aed0cdcbb87f") : secret "maas-api-e2e-ipp-a-69ef21-serving-cert" not found
Sep 06 10:53:38.313864 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.313845    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8\" (UID: \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"
Sep 06 10:53:38.315737 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.315715    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-metrics-certs\") pod \"maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8\" (UID: \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"
Sep 06 10:53:38.325816 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.325789    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdqzq\" (UniqueName: \"kubernetes.io/projected/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-kube-api-access-jdqzq\") pod \"maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8\" (UID: \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"
Sep 06 10:53:38.817771 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.817731    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8\" (UID: \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"
Sep 06 10:53:38.820554 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:38.820525    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8\" (UID: \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"
Sep 06 10:53:39.048043 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:39.048011    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"
Sep 06 10:53:39.320681 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:39.320657    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"]
Sep 06 10:53:39.322863 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:53:39.322835    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31d11217_4f66_4a6e_99f4_aed0cdcbb87f.slice/crio-0861c0c724ac15e7be656114c946b0c46c92309f6da96b14ec4e5c71471b3e87 WatchSource:0}: Error finding container 0861c0c724ac15e7be656114c946b0c46c92309f6da96b14ec4e5c71471b3e87: Status 404 returned error can't find the container with id 0861c0c724ac15e7be656114c946b0c46c92309f6da96b14ec4e5c71471b3e87
Sep 06 10:53:39.324516 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:39.324484    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 10:53:40.043833 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:40.043748    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8" event={"ID":"31d11217-4f66-4a6e-99f4-aed0cdcbb87f","Type":"ContainerStarted","Data":"b8308c8c1b335977b2acd71011c6d35a75dd562a29b7e7affd0d4a5fedb4ca6b"}
Sep 06 10:53:40.043833 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:40.043810    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"
Sep 06 10:53:40.044015 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:40.043834    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8" event={"ID":"31d11217-4f66-4a6e-99f4-aed0cdcbb87f","Type":"ContainerStarted","Data":"0861c0c724ac15e7be656114c946b0c46c92309f6da96b14ec4e5c71471b3e87"}
Sep 06 10:53:40.060205 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:40.060153    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8" podStartSLOduration=1.700454583 podStartE2EDuration="2.060139244s" totalImagesPullingTime="359.684661ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:53:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:53:40.058466686 +0000 UTC m=+1432.508247454" watchObservedRunningTime="2026-09-06 10:53:40.060139244 +0000 UTC m=+1432.509920012"
Sep 06 10:53:46.054396 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:46.054317    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"
Sep 06 10:53:48.422097 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:48.422060    2557 scope.go:117] "RemoveContainer" containerID="7b5075514643807929caf7ec8a55b0b8d66cd666366774c1c6cd50e4ea1e20f3"
Sep 06 10:53:48.430566 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:48.430546    2557 scope.go:117] "RemoveContainer" containerID="da0037da7dee09fdb35b66bc4fec346de13c9a850735ea1baa6b61583ede2ac0"
Sep 06 10:53:49.420813 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:49.420774    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:53:51.330013 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:51.329223    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": read tcp 10.132.0.2:35098->10.132.0.81:15021: read: connection reset by peer" start-of-body=
Sep 06 10:53:51.330013 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:51.329310    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n" podUID="4dc570f5-e559-48cc-8dcd-95fcae8ee57f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": read tcp 10.132.0.2:35098->10.132.0.81:15021: read: connection reset by peer"
Sep 06 10:53:52.117512 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:52.117463    2557 generic.go:356] "Generic (PLEG): container finished" podID="4dc570f5-e559-48cc-8dcd-95fcae8ee57f" containerID="f229c1ca8ee823546195a6a5a4b11a73897e93b5d9ef21c96f9f685cd262688e" exitCode=0
Sep 06 10:53:52.117702 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:52.117543    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n" event={"ID":"4dc570f5-e559-48cc-8dcd-95fcae8ee57f","Type":"ContainerDied","Data":"f229c1ca8ee823546195a6a5a4b11a73897e93b5d9ef21c96f9f685cd262688e"}
Sep 06 10:53:52.117952 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:52.117938    2557 scope.go:117] "RemoveContainer" containerID="f229c1ca8ee823546195a6a5a4b11a73897e93b5d9ef21c96f9f685cd262688e"
Sep 06 10:53:52.119065 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:52.119038    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:53:52.119121 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:52.119105    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:53:52.119164 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:52.119138    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:53:53.125093 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:53.125058    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n" event={"ID":"4dc570f5-e559-48cc-8dcd-95fcae8ee57f","Type":"ContainerStarted","Data":"c0311964470312f381ca3b284a7eba05a362964b4ec3a88d68f3a16f8cad99a5"}
Sep 06 10:53:53.860721 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:53.860675    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:53.860935 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:53.860911    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 06 10:53:53.861031 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:53.861004    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n" podUID="4dc570f5-e559-48cc-8dcd-95fcae8ee57f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 06 10:53:54.860512 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:54.860449    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 06 10:53:54.860908 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:54.860539    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n" podUID="4dc570f5-e559-48cc-8dcd-95fcae8ee57f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 06 10:53:55.860794 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:55.860758    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 06 10:53:55.861192 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:55.860826    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n" podUID="4dc570f5-e559-48cc-8dcd-95fcae8ee57f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 06 10:53:56.860552 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:56.860513    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 06 10:53:56.860856 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:56.860575    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n" podUID="4dc570f5-e559-48cc-8dcd-95fcae8ee57f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 06 10:53:57.861168 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:57.861127    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 06 10:53:57.861587 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:57.861197    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n" podUID="4dc570f5-e559-48cc-8dcd-95fcae8ee57f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 06 10:53:58.863683 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:58.863648    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:58.864057 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:58.863907    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:53:58.864568 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:53:58.864546    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:54:02.864786 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:02.864752    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"]
Sep 06 10:54:02.867833 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:02.867811    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:02.870122 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:02.870100    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-8c14da-openshift-default-dockercfg-8r5mf\""
Sep 06 10:54:02.879082 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:02.879056    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"]
Sep 06 10:54:02.971756 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:02.971719    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-envoy\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:02.971945 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:02.971813    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-data\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:02.971945 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:02.971851    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrh8s\" (UniqueName: \"kubernetes.io/projected/0faaade3-425c-4452-a3ea-a23a7219e8ae-kube-api-access-lrh8s\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:02.972046 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:02.971934    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-token\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:02.972046 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:02.971990    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-workload-certs\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:02.972046 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:02.972019    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0faaade3-425c-4452-a3ea-a23a7219e8ae-istiod-ca-cert\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:02.972156 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:02.972060    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-credential-socket\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:02.972156 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:02.972111    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-podinfo\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:02.972156 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:02.972137    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-workload-socket\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.073452 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.073415    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-token\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.073452 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.073456    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-workload-certs\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.073735 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.073516    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0faaade3-425c-4452-a3ea-a23a7219e8ae-istiod-ca-cert\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.073735 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.073578    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-credential-socket\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.073735 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.073653    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-podinfo\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.073735 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.073687    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-workload-socket\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.073959 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.073776    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-envoy\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.073959 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.073811    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-data\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.073959 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.073836    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrh8s\" (UniqueName: \"kubernetes.io/projected/0faaade3-425c-4452-a3ea-a23a7219e8ae-kube-api-access-lrh8s\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.073959 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.073893    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-workload-certs\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.074183 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.074150    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-credential-socket\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.074273 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.074240    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-workload-socket\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.074350 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.074295    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0faaade3-425c-4452-a3ea-a23a7219e8ae-istiod-ca-cert\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.074350 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.074325    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-data\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.076276 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.076250    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-envoy\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.076388 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.076343    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-podinfo\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.081152 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.081122    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrh8s\" (UniqueName: \"kubernetes.io/projected/0faaade3-425c-4452-a3ea-a23a7219e8ae-kube-api-access-lrh8s\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.081549 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.081531    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-token\") pod \"e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") " pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.183057 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.182973    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:03.459750 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.459725    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"]
Sep 06 10:54:03.461152 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:54:03.461118    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0faaade3_425c_4452_a3ea_a23a7219e8ae.slice/crio-5c5c15cf7867a3accefbda7fcfeecbbe47068c22405c0b88894f2ca45940f924 WatchSource:0}: Error finding container 5c5c15cf7867a3accefbda7fcfeecbbe47068c22405c0b88894f2ca45940f924: Status 404 returned error can't find the container with id 5c5c15cf7867a3accefbda7fcfeecbbe47068c22405c0b88894f2ca45940f924
Sep 06 10:54:03.464094 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.464059    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:54:03.464170 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.464129    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:54:03.464170 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:03.464156    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:54:04.182873 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:04.182836    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh" event={"ID":"0faaade3-425c-4452-a3ea-a23a7219e8ae","Type":"ContainerStarted","Data":"372431e7cbfe7ef0e679449918c44523538c73a137070734a9294a8b30ccd362"}
Sep 06 10:54:04.183327 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:04.182901    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh" event={"ID":"0faaade3-425c-4452-a3ea-a23a7219e8ae","Type":"ContainerStarted","Data":"5c5c15cf7867a3accefbda7fcfeecbbe47068c22405c0b88894f2ca45940f924"}
Sep 06 10:54:04.183327 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:04.183091    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:04.188362 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:04.188342    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:04.201932 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:04.201884    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh" podStartSLOduration=2.201869672 podStartE2EDuration="2.201869672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:54:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:54:04.200731875 +0000 UTC m=+1456.650512656" watchObservedRunningTime="2026-09-06 10:54:04.201869672 +0000 UTC m=+1456.651650439"
Sep 06 10:54:05.194199 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:05.194161    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:05.195097 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:05.195075    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:07.236661 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:07.236619    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"]
Sep 06 10:54:07.239939 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:07.239917    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"
Sep 06 10:54:07.242198 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:07.242173    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-8c14da-serving-cert\""
Sep 06 10:54:07.251942 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:07.251916    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"]
Sep 06 10:54:07.312422 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:07.312391    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-metrics-certs\") pod \"maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c\" (UID: \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"
Sep 06 10:54:07.312593 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:07.312432    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-maas-api-tls\") pod \"maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c\" (UID: \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"
Sep 06 10:54:07.312593 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:07.312556    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z9cc\" (UniqueName: \"kubernetes.io/projected/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-kube-api-access-6z9cc\") pod \"maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c\" (UID: \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"
Sep 06 10:54:07.312698 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:07.312615    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c\" (UID: \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"
Sep 06 10:54:07.414021 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:07.413974    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-metrics-certs\") pod \"maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c\" (UID: \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"
Sep 06 10:54:07.414021 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:07.414030    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-maas-api-tls\") pod \"maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c\" (UID: \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"
Sep 06 10:54:07.414293 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:07.414055    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z9cc\" (UniqueName: \"kubernetes.io/projected/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-kube-api-access-6z9cc\") pod \"maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c\" (UID: \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"
Sep 06 10:54:07.414293 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:07.414088    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c\" (UID: \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"
Sep 06 10:54:07.414293 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:54:07.414190    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-serving-cert: secret "maas-api-e2e-ipp-b-8c14da-serving-cert" not found
Sep 06 10:54:07.414293 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:54:07.414287    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-maas-api-tls podName:a98f8662-1af8-4697-94ea-ff5a3a4f8a57 nodeName:}" failed. No retries permitted until 2026-09-06 10:54:07.914266455 +0000 UTC m=+1460.364047227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-maas-api-tls") pod "maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c" (UID: "a98f8662-1af8-4697-94ea-ff5a3a4f8a57") : secret "maas-api-e2e-ipp-b-8c14da-serving-cert" not found
Sep 06 10:54:07.414826 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:07.414808    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c\" (UID: \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"
Sep 06 10:54:07.416419 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:07.416402    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-metrics-certs\") pod \"maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c\" (UID: \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"
Sep 06 10:54:07.422049 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:07.422025    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z9cc\" (UniqueName: \"kubernetes.io/projected/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-kube-api-access-6z9cc\") pod \"maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c\" (UID: \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"
Sep 06 10:54:07.920208 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:07.920178    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-maas-api-tls\") pod \"maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c\" (UID: \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"
Sep 06 10:54:07.922552 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:07.922529    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-maas-api-tls\") pod \"maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c\" (UID: \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"
Sep 06 10:54:08.153656 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:08.153626    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"
Sep 06 10:54:08.425291 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:08.425254    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"]
Sep 06 10:54:08.426971 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:54:08.426941    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda98f8662_1af8_4697_94ea_ff5a3a4f8a57.slice/crio-538b9893f0a78ea9600af6e6174739582850478c5f3fb13ddbdae39ca612a0c5 WatchSource:0}: Error finding container 538b9893f0a78ea9600af6e6174739582850478c5f3fb13ddbdae39ca612a0c5: Status 404 returned error can't find the container with id 538b9893f0a78ea9600af6e6174739582850478c5f3fb13ddbdae39ca612a0c5
Sep 06 10:54:09.221696 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:09.221665    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c" event={"ID":"a98f8662-1af8-4697-94ea-ff5a3a4f8a57","Type":"ContainerStarted","Data":"d72a6377290e4c820d13fc7b9430d8a57b9293fd5bf8e136bceb2f39686335ab"}
Sep 06 10:54:09.221866 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:09.221720    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c" event={"ID":"a98f8662-1af8-4697-94ea-ff5a3a4f8a57","Type":"ContainerStarted","Data":"538b9893f0a78ea9600af6e6174739582850478c5f3fb13ddbdae39ca612a0c5"}
Sep 06 10:54:09.221866 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:09.221742    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"
Sep 06 10:54:09.240880 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:09.240832    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c" podStartSLOduration=1.642894061 podStartE2EDuration="2.240815738s" totalImagesPullingTime="597.921677ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:54:09.238197144 +0000 UTC m=+1461.687977911" watchObservedRunningTime="2026-09-06 10:54:09.240815738 +0000 UTC m=+1461.690596546"
Sep 06 10:54:11.818028 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:11.817987    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:54:15.232485 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:15.232459    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"
Sep 06 10:54:21.236685 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:21.236641    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 06 10:54:21.237094 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:21.236739    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh" podUID="0faaade3-425c-4452-a3ea-a23a7219e8ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 06 10:54:22.298470 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:22.298432    2557 generic.go:356] "Generic (PLEG): container finished" podID="0faaade3-425c-4452-a3ea-a23a7219e8ae" containerID="372431e7cbfe7ef0e679449918c44523538c73a137070734a9294a8b30ccd362" exitCode=0
Sep 06 10:54:22.298914 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:22.298518    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh" event={"ID":"0faaade3-425c-4452-a3ea-a23a7219e8ae","Type":"ContainerDied","Data":"372431e7cbfe7ef0e679449918c44523538c73a137070734a9294a8b30ccd362"}
Sep 06 10:54:22.298914 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:22.298898    2557 scope.go:117] "RemoveContainer" containerID="372431e7cbfe7ef0e679449918c44523538c73a137070734a9294a8b30ccd362"
Sep 06 10:54:22.299940 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:22.299908    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:54:22.300047 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:22.299988    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:54:22.300047 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:22.300036    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:54:23.306352 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:23.306309    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh" event={"ID":"0faaade3-425c-4452-a3ea-a23a7219e8ae","Type":"ContainerStarted","Data":"5c0d73fc15e5ae8cb4ed9b1a24b0e286c1b41250018b5b34c865aa465eed5da3"}
Sep 06 10:54:24.184226 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:24.184183    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:24.184533 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:24.184259    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 06 10:54:24.184775 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:24.184748    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh" podUID="0faaade3-425c-4452-a3ea-a23a7219e8ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 06 10:54:25.186868 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:25.186831    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 06 10:54:25.187842 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:25.187411    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh" podUID="0faaade3-425c-4452-a3ea-a23a7219e8ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 06 10:54:26.184358 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:26.183904    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 06 10:54:26.184358 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:26.183974    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh" podUID="0faaade3-425c-4452-a3ea-a23a7219e8ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 06 10:54:27.184158 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:27.184114    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 06 10:54:27.184772 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:27.184741    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh" podUID="0faaade3-425c-4452-a3ea-a23a7219e8ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 06 10:54:28.183890 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:28.183850    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 06 10:54:28.184190 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:28.184162    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh" podUID="0faaade3-425c-4452-a3ea-a23a7219e8ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 06 10:54:29.187205 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:29.187171    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:29.187673 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:29.187515    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:29.188146 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:29.188128    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:54:48.126657 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:48.126628    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/2.log"
Sep 06 10:54:48.135387 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:48.135366    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/2.log"
Sep 06 10:54:49.720280 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:54:49.720246    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:55:08.919161 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:08.919120    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:55:12.598250 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.598174    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"]
Sep 06 10:55:12.601749 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.601728    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.604251 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.604228    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-676ab2-openshift-default-dockercfg-hqmdp\""
Sep 06 10:55:12.612391 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.612366    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"]
Sep 06 10:55:12.756540 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.756505    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-envoy\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.756730 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.756553    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-token\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.756730 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.756600    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-data\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.756730 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.756672    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istiod-ca-cert\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.756730 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.756692    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-credential-socket\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.756730 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.756711    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-workload-certs\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.756937 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.756752    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-workload-socket\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.756937 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.756803    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-podinfo\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.756937 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.756846    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-949g2\" (UniqueName: \"kubernetes.io/projected/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-kube-api-access-949g2\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.858429 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.858291    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-data\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.858603 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.858437    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istiod-ca-cert\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.858603 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.858457    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-credential-socket\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.858603 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.858481    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-workload-certs\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.858603 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.858548    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-workload-socket\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.858809 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.858601    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-podinfo\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.858809 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.858616    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-data\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.858809 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.858640    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-949g2\" (UniqueName: \"kubernetes.io/projected/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-kube-api-access-949g2\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.858809 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.858698    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-envoy\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.858809 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.858740    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-token\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.859079 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.858813    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-credential-socket\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.859079 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.858867    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-workload-certs\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.859079 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.858894    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-workload-socket\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.859188 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.859102    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istiod-ca-cert\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.861003 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.860971    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-envoy\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.861178 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.861157    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-podinfo\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.866224 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.866202    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-949g2\" (UniqueName: \"kubernetes.io/projected/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-kube-api-access-949g2\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.866375 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.866289    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-token\") pod \"e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") " pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:12.914175 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:12.914060    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:13.187641 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:13.187612    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"]
Sep 06 10:55:13.190074 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:55:13.190043    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e44fe83_c490_4c3f_bfd8_b4821bf34fcd.slice/crio-c26b09823009788557cc7c23c1979fd272c0c452b0060e134fb7a478d594d96b WatchSource:0}: Error finding container c26b09823009788557cc7c23c1979fd272c0c452b0060e134fb7a478d594d96b: Status 404 returned error can't find the container with id c26b09823009788557cc7c23c1979fd272c0c452b0060e134fb7a478d594d96b
Sep 06 10:55:13.192237 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:13.192201    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:55:13.192338 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:13.192268    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:55:13.192338 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:13.192298    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:55:13.589271 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:13.589241    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22" event={"ID":"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd","Type":"ContainerStarted","Data":"883931f7262dda5671fc7dc562b7d66b1d335fc1360fe34ebae6eded673759ff"}
Sep 06 10:55:13.589433 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:13.589293    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22" event={"ID":"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd","Type":"ContainerStarted","Data":"c26b09823009788557cc7c23c1979fd272c0c452b0060e134fb7a478d594d96b"}
Sep 06 10:55:13.608328 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:13.608279    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22" podStartSLOduration=1.608262273 podStartE2EDuration="1.608262273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:55:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:55:13.606835882 +0000 UTC m=+1526.056616650" watchObservedRunningTime="2026-09-06 10:55:13.608262273 +0000 UTC m=+1526.058043045"
Sep 06 10:55:13.915298 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:13.915220    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:14.926704 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:14.926670    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:15.600653 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:15.600626    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:15.601549 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:15.601529    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:20.918797 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:20.918761    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:55:31.290316 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:31.289057    2557 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": read tcp 10.132.0.2:42200->10.132.0.85:15021: read: connection reset by peer" start-of-body=
Sep 06 10:55:31.290316 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:31.289127    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22" podUID="7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": read tcp 10.132.0.2:42200->10.132.0.85:15021: read: connection reset by peer"
Sep 06 10:55:31.686588 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:31.686553    2557 generic.go:356] "Generic (PLEG): container finished" podID="7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" containerID="883931f7262dda5671fc7dc562b7d66b1d335fc1360fe34ebae6eded673759ff" exitCode=0
Sep 06 10:55:31.686765 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:31.686630    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22" event={"ID":"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd","Type":"ContainerDied","Data":"883931f7262dda5671fc7dc562b7d66b1d335fc1360fe34ebae6eded673759ff"}
Sep 06 10:55:31.687056 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:31.687042    2557 scope.go:117] "RemoveContainer" containerID="883931f7262dda5671fc7dc562b7d66b1d335fc1360fe34ebae6eded673759ff"
Sep 06 10:55:31.688258 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:31.688232    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:55:31.688320 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:31.688299    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:55:31.688357 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:31.688329    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 06 10:55:32.696707 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:32.696671    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22" event={"ID":"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd","Type":"ContainerStarted","Data":"95c410a0cbdaad2a0d2915fba526e8a5c8a0493bf64db9216d9e213d728a0bfe"}
Sep 06 10:55:32.915263 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:32.915188    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:32.915523 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:32.915403    2557 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 06 10:55:32.915632 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:32.915559    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22" podUID="7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 06 10:55:33.915033 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:33.914998    2557 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 06 10:55:33.915411 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:33.915064    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22" podUID="7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 06 10:55:34.915374 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:34.915337    2557 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 06 10:55:34.915794 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:34.915395    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22" podUID="7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 06 10:55:35.915183 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:35.915148    2557 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 06 10:55:35.915362 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:35.915205    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22" podUID="7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 06 10:55:36.914702 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:36.914663    2557 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 06 10:55:36.915087 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:36.914727    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22" podUID="7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 06 10:55:38.075757 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:38.075724    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:38.076133 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:38.076110    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:38.076875 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:38.076853    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:51.111327 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:51.111286    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:55:53.710182 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:53.710151    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"]
Sep 06 10:55:53.710681 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:53.710407    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22" podUID="7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" containerName="istio-proxy" containerID="cri-o://95c410a0cbdaad2a0d2915fba526e8a5c8a0493bf64db9216d9e213d728a0bfe" gracePeriod=30
Sep 06 10:55:58.843622 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:58.843586    2557 generic.go:356] "Generic (PLEG): container finished" podID="7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" containerID="95c410a0cbdaad2a0d2915fba526e8a5c8a0493bf64db9216d9e213d728a0bfe" exitCode=0
Sep 06 10:55:58.843977 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:58.843663    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22" event={"ID":"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd","Type":"ContainerDied","Data":"95c410a0cbdaad2a0d2915fba526e8a5c8a0493bf64db9216d9e213d728a0bfe"}
Sep 06 10:55:58.843977 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:58.843740    2557 scope.go:117] "RemoveContainer" containerID="883931f7262dda5671fc7dc562b7d66b1d335fc1360fe34ebae6eded673759ff"
Sep 06 10:55:58.962258 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:58.962230    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:59.028988 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.028913    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-credential-socket\") pod \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") "
Sep 06 10:55:59.028988 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.028948    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istiod-ca-cert\") pod \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") "
Sep 06 10:55:59.028988 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.028977    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-data\") pod \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") "
Sep 06 10:55:59.029260 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.028992    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-token\") pod \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") "
Sep 06 10:55:59.029260 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.029036    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-podinfo\") pod \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") "
Sep 06 10:55:59.029260 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.029090    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-949g2\" (UniqueName: \"kubernetes.io/projected/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-kube-api-access-949g2\") pod \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") "
Sep 06 10:55:59.029260 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.029158    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-workload-socket\") pod \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") "
Sep 06 10:55:59.029260 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.029180    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-envoy\") pod \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") "
Sep 06 10:55:59.029260 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.029236    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-workload-certs\") pod \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\" (UID: \"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd\") "
Sep 06 10:55:59.029260 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.029235    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" (UID: "7e44fe83-c490-4c3f-bfd8-b4821bf34fcd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:55:59.029677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.029403    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" (UID: "7e44fe83-c490-4c3f-bfd8-b4821bf34fcd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:55:59.029677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.029460    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" (UID: "7e44fe83-c490-4c3f-bfd8-b4821bf34fcd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:55:59.029677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.029601    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-data" (OuterVolumeSpecName: "istio-data") pod "7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" (UID: "7e44fe83-c490-4c3f-bfd8-b4821bf34fcd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:55:59.029677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.029612    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" (UID: "7e44fe83-c490-4c3f-bfd8-b4821bf34fcd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:55:59.029903 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.029707    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-workload-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:55:59.029903 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.029727    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-workload-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:55:59.029903 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.029740    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-credential-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:55:59.029903 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.029754    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istiod-ca-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:55:59.029903 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.029767    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-data\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:55:59.031232 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.031210    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-token" (OuterVolumeSpecName: "istio-token") pod "7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" (UID: "7e44fe83-c490-4c3f-bfd8-b4821bf34fcd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:55:59.031340 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.031260    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-kube-api-access-949g2" (OuterVolumeSpecName: "kube-api-access-949g2") pod "7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" (UID: "7e44fe83-c490-4c3f-bfd8-b4821bf34fcd"). InnerVolumeSpecName "kube-api-access-949g2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:55:59.031387 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.031339    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" (UID: "7e44fe83-c490-4c3f-bfd8-b4821bf34fcd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:55:59.031425 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.031398    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" (UID: "7e44fe83-c490-4c3f-bfd8-b4821bf34fcd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 10:55:59.130547 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.130519    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-envoy\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:55:59.130547 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.130542    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:55:59.130547 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.130550    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-istio-podinfo\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:55:59.130758 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.130559    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-949g2\" (UniqueName: \"kubernetes.io/projected/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd-kube-api-access-949g2\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:55:59.851066 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.851036    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"
Sep 06 10:55:59.851534 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.851077    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22" event={"ID":"7e44fe83-c490-4c3f-bfd8-b4821bf34fcd","Type":"ContainerDied","Data":"c26b09823009788557cc7c23c1979fd272c0c452b0060e134fb7a478d594d96b"}
Sep 06 10:55:59.851534 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.851143    2557 scope.go:117] "RemoveContainer" containerID="95c410a0cbdaad2a0d2915fba526e8a5c8a0493bf64db9216d9e213d728a0bfe"
Sep 06 10:55:59.872552 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.872523    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"]
Sep 06 10:55:59.875768 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:55:59.875746    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-676ab2-openshift-default-54b7446bbd-8jz22"]
Sep 06 10:56:00.036691 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:00.036659    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" path="/var/lib/kubelet/pods/7e44fe83-c490-4c3f-bfd8-b4821bf34fcd/volumes"
Sep 06 10:56:10.316725 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:10.316689    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:56:12.655046 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:12.655012    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"]
Sep 06 10:56:12.926108 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:12.926007    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8" podUID="31d11217-4f66-4a6e-99f4-aed0cdcbb87f" containerName="maas-api" containerID="cri-o://b8308c8c1b335977b2acd71011c6d35a75dd562a29b7e7affd0d4a5fedb4ca6b" gracePeriod=30
Sep 06 10:56:13.173417 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.173391    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"
Sep 06 10:56:13.262836 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.262743    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-metrics-certs\") pod \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\" (UID: \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\") "
Sep 06 10:56:13.262836 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.262813    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdqzq\" (UniqueName: \"kubernetes.io/projected/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-kube-api-access-jdqzq\") pod \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\" (UID: \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\") "
Sep 06 10:56:13.262836 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.262843    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-openshift-service-ca\") pod \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\" (UID: \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\") "
Sep 06 10:56:13.263118 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.262919    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-maas-api-tls\") pod \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\" (UID: \"31d11217-4f66-4a6e-99f4-aed0cdcbb87f\") "
Sep 06 10:56:13.263283 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.263256    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "31d11217-4f66-4a6e-99f4-aed0cdcbb87f" (UID: "31d11217-4f66-4a6e-99f4-aed0cdcbb87f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:56:13.265004 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.264969    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "31d11217-4f66-4a6e-99f4-aed0cdcbb87f" (UID: "31d11217-4f66-4a6e-99f4-aed0cdcbb87f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 10:56:13.265004 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.264995    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "31d11217-4f66-4a6e-99f4-aed0cdcbb87f" (UID: "31d11217-4f66-4a6e-99f4-aed0cdcbb87f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 10:56:13.265181 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.265091    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-kube-api-access-jdqzq" (OuterVolumeSpecName: "kube-api-access-jdqzq") pod "31d11217-4f66-4a6e-99f4-aed0cdcbb87f" (UID: "31d11217-4f66-4a6e-99f4-aed0cdcbb87f"). InnerVolumeSpecName "kube-api-access-jdqzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:56:13.364356 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.364324    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-maas-api-tls\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:56:13.364356 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.364350    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-metrics-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:56:13.364356 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.364360    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdqzq\" (UniqueName: \"kubernetes.io/projected/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-kube-api-access-jdqzq\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:56:13.364662 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.364369    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31d11217-4f66-4a6e-99f4-aed0cdcbb87f-openshift-service-ca\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:56:13.936381 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.936347    2557 generic.go:356] "Generic (PLEG): container finished" podID="31d11217-4f66-4a6e-99f4-aed0cdcbb87f" containerID="b8308c8c1b335977b2acd71011c6d35a75dd562a29b7e7affd0d4a5fedb4ca6b" exitCode=0
Sep 06 10:56:13.936863 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.936397    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"
Sep 06 10:56:13.936863 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.936425    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8" event={"ID":"31d11217-4f66-4a6e-99f4-aed0cdcbb87f","Type":"ContainerDied","Data":"b8308c8c1b335977b2acd71011c6d35a75dd562a29b7e7affd0d4a5fedb4ca6b"}
Sep 06 10:56:13.936863 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.936526    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8" event={"ID":"31d11217-4f66-4a6e-99f4-aed0cdcbb87f","Type":"ContainerDied","Data":"0861c0c724ac15e7be656114c946b0c46c92309f6da96b14ec4e5c71471b3e87"}
Sep 06 10:56:13.936863 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.936553    2557 scope.go:117] "RemoveContainer" containerID="b8308c8c1b335977b2acd71011c6d35a75dd562a29b7e7affd0d4a5fedb4ca6b"
Sep 06 10:56:13.945989 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.945965    2557 scope.go:117] "RemoveContainer" containerID="b8308c8c1b335977b2acd71011c6d35a75dd562a29b7e7affd0d4a5fedb4ca6b"
Sep 06 10:56:13.946267 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:56:13.946246    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8308c8c1b335977b2acd71011c6d35a75dd562a29b7e7affd0d4a5fedb4ca6b\": container with ID starting with b8308c8c1b335977b2acd71011c6d35a75dd562a29b7e7affd0d4a5fedb4ca6b not found: ID does not exist" containerID="b8308c8c1b335977b2acd71011c6d35a75dd562a29b7e7affd0d4a5fedb4ca6b"
Sep 06 10:56:13.946343 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.946278    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8308c8c1b335977b2acd71011c6d35a75dd562a29b7e7affd0d4a5fedb4ca6b"} err="failed to get container status \"b8308c8c1b335977b2acd71011c6d35a75dd562a29b7e7affd0d4a5fedb4ca6b\": rpc error: code = NotFound desc = could not find container \"b8308c8c1b335977b2acd71011c6d35a75dd562a29b7e7affd0d4a5fedb4ca6b\": container with ID starting with b8308c8c1b335977b2acd71011c6d35a75dd562a29b7e7affd0d4a5fedb4ca6b not found: ID does not exist"
Sep 06 10:56:13.957673 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.957653    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"]
Sep 06 10:56:13.961251 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:13.961230    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-69ef21-7d4cf6b7b6-rnzx8"]
Sep 06 10:56:14.036864 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:14.036835    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31d11217-4f66-4a6e-99f4-aed0cdcbb87f" path="/var/lib/kubelet/pods/31d11217-4f66-4a6e-99f4-aed0cdcbb87f/volumes"
Sep 06 10:56:23.514003 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:23.513968    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:56:48.480411 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:56:48.480331    2557 scope.go:117] "RemoveContainer" containerID="4f3f62c56b690813e92b8c18cf418887851e3a0731d09976f3cb1c2afed4e2a7"
Sep 06 10:57:06.409308 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:06.409265    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"]
Sep 06 10:57:06.410271 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:06.410236    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n" podUID="4dc570f5-e559-48cc-8dcd-95fcae8ee57f" containerName="istio-proxy" containerID="cri-o://c0311964470312f381ca3b284a7eba05a362964b4ec3a88d68f3a16f8cad99a5" gracePeriod=30
Sep 06 10:57:11.670308 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.670230    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:57:11.812541 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.812456    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-envoy\") pod \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") "
Sep 06 10:57:11.812541 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.812545    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istiod-ca-cert\") pod \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") "
Sep 06 10:57:11.812788 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.812568    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psqdr\" (UniqueName: \"kubernetes.io/projected/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-kube-api-access-psqdr\") pod \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") "
Sep 06 10:57:11.812788 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.812617    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-podinfo\") pod \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") "
Sep 06 10:57:11.812788 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.812646    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-token\") pod \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") "
Sep 06 10:57:11.812788 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.812670    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-data\") pod \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") "
Sep 06 10:57:11.812788 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.812692    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-workload-certs\") pod \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") "
Sep 06 10:57:11.812788 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.812735    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-credential-socket\") pod \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") "
Sep 06 10:57:11.813124 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.812796    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-workload-socket\") pod \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\" (UID: \"4dc570f5-e559-48cc-8dcd-95fcae8ee57f\") "
Sep 06 10:57:11.813124 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.813001    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-data" (OuterVolumeSpecName: "istio-data") pod "4dc570f5-e559-48cc-8dcd-95fcae8ee57f" (UID: "4dc570f5-e559-48cc-8dcd-95fcae8ee57f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:57:11.813239 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.813212    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-data\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:11.813239 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.813017    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4dc570f5-e559-48cc-8dcd-95fcae8ee57f" (UID: "4dc570f5-e559-48cc-8dcd-95fcae8ee57f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:57:11.813345 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.813246    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4dc570f5-e559-48cc-8dcd-95fcae8ee57f" (UID: "4dc570f5-e559-48cc-8dcd-95fcae8ee57f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:57:11.813345 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.813290    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4dc570f5-e559-48cc-8dcd-95fcae8ee57f" (UID: "4dc570f5-e559-48cc-8dcd-95fcae8ee57f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:57:11.813345 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.813330    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4dc570f5-e559-48cc-8dcd-95fcae8ee57f" (UID: "4dc570f5-e559-48cc-8dcd-95fcae8ee57f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:57:11.814910 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.814881    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4dc570f5-e559-48cc-8dcd-95fcae8ee57f" (UID: "4dc570f5-e559-48cc-8dcd-95fcae8ee57f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 10:57:11.815049 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.815027    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-kube-api-access-psqdr" (OuterVolumeSpecName: "kube-api-access-psqdr") pod "4dc570f5-e559-48cc-8dcd-95fcae8ee57f" (UID: "4dc570f5-e559-48cc-8dcd-95fcae8ee57f"). InnerVolumeSpecName "kube-api-access-psqdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:57:11.815190 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.815167    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4dc570f5-e559-48cc-8dcd-95fcae8ee57f" (UID: "4dc570f5-e559-48cc-8dcd-95fcae8ee57f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:57:11.815252 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.815171    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-token" (OuterVolumeSpecName: "istio-token") pod "4dc570f5-e559-48cc-8dcd-95fcae8ee57f" (UID: "4dc570f5-e559-48cc-8dcd-95fcae8ee57f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:57:11.914046 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.913988    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-credential-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:11.914046 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.914038    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-workload-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:11.914046 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.914054    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-envoy\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:11.914310 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.914066    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istiod-ca-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:11.914310 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.914080    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psqdr\" (UniqueName: \"kubernetes.io/projected/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-kube-api-access-psqdr\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:11.914310 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.914092    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-podinfo\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:11.914310 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.914104    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-istio-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:11.914310 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:11.914115    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc570f5-e559-48cc-8dcd-95fcae8ee57f-workload-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:12.245918 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:12.245876    2557 generic.go:356] "Generic (PLEG): container finished" podID="4dc570f5-e559-48cc-8dcd-95fcae8ee57f" containerID="c0311964470312f381ca3b284a7eba05a362964b4ec3a88d68f3a16f8cad99a5" exitCode=0
Sep 06 10:57:12.246114 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:12.245930    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"
Sep 06 10:57:12.246114 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:12.245920    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n" event={"ID":"4dc570f5-e559-48cc-8dcd-95fcae8ee57f","Type":"ContainerDied","Data":"c0311964470312f381ca3b284a7eba05a362964b4ec3a88d68f3a16f8cad99a5"}
Sep 06 10:57:12.246114 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:12.246066    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n" event={"ID":"4dc570f5-e559-48cc-8dcd-95fcae8ee57f","Type":"ContainerDied","Data":"466221e7f47b5cf120bf709b412e7964d79ea021f38ba545dddac865753e83a5"}
Sep 06 10:57:12.246114 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:12.246097    2557 scope.go:117] "RemoveContainer" containerID="c0311964470312f381ca3b284a7eba05a362964b4ec3a88d68f3a16f8cad99a5"
Sep 06 10:57:12.255072 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:12.255054    2557 scope.go:117] "RemoveContainer" containerID="f229c1ca8ee823546195a6a5a4b11a73897e93b5d9ef21c96f9f685cd262688e"
Sep 06 10:57:12.262439 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:12.262417    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"]
Sep 06 10:57:12.263864 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:12.263844    2557 scope.go:117] "RemoveContainer" containerID="c0311964470312f381ca3b284a7eba05a362964b4ec3a88d68f3a16f8cad99a5"
Sep 06 10:57:12.264223 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:57:12.264188    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0311964470312f381ca3b284a7eba05a362964b4ec3a88d68f3a16f8cad99a5\": container with ID starting with c0311964470312f381ca3b284a7eba05a362964b4ec3a88d68f3a16f8cad99a5 not found: ID does not exist" containerID="c0311964470312f381ca3b284a7eba05a362964b4ec3a88d68f3a16f8cad99a5"
Sep 06 10:57:12.264332 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:12.264232    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0311964470312f381ca3b284a7eba05a362964b4ec3a88d68f3a16f8cad99a5"} err="failed to get container status \"c0311964470312f381ca3b284a7eba05a362964b4ec3a88d68f3a16f8cad99a5\": rpc error: code = NotFound desc = could not find container \"c0311964470312f381ca3b284a7eba05a362964b4ec3a88d68f3a16f8cad99a5\": container with ID starting with c0311964470312f381ca3b284a7eba05a362964b4ec3a88d68f3a16f8cad99a5 not found: ID does not exist"
Sep 06 10:57:12.264332 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:12.264258    2557 scope.go:117] "RemoveContainer" containerID="f229c1ca8ee823546195a6a5a4b11a73897e93b5d9ef21c96f9f685cd262688e"
Sep 06 10:57:12.264576 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:57:12.264555    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f229c1ca8ee823546195a6a5a4b11a73897e93b5d9ef21c96f9f685cd262688e\": container with ID starting with f229c1ca8ee823546195a6a5a4b11a73897e93b5d9ef21c96f9f685cd262688e not found: ID does not exist" containerID="f229c1ca8ee823546195a6a5a4b11a73897e93b5d9ef21c96f9f685cd262688e"
Sep 06 10:57:12.264646 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:12.264581    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f229c1ca8ee823546195a6a5a4b11a73897e93b5d9ef21c96f9f685cd262688e"} err="failed to get container status \"f229c1ca8ee823546195a6a5a4b11a73897e93b5d9ef21c96f9f685cd262688e\": rpc error: code = NotFound desc = could not find container \"f229c1ca8ee823546195a6a5a4b11a73897e93b5d9ef21c96f9f685cd262688e\": container with ID starting with f229c1ca8ee823546195a6a5a4b11a73897e93b5d9ef21c96f9f685cd262688e not found: ID does not exist"
Sep 06 10:57:12.266174 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:12.266152    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-69ef21-openshift-default-5dddfbddcb-4wc6n"]
Sep 06 10:57:14.036834 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:14.036799    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dc570f5-e559-48cc-8dcd-95fcae8ee57f" path="/var/lib/kubelet/pods/4dc570f5-e559-48cc-8dcd-95fcae8ee57f/volumes"
Sep 06 10:57:17.204540 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:17.204503    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"]
Sep 06 10:57:17.204995 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:17.204795    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c" podUID="a98f8662-1af8-4697-94ea-ff5a3a4f8a57" containerName="maas-api" containerID="cri-o://d72a6377290e4c820d13fc7b9430d8a57b9293fd5bf8e136bceb2f39686335ab" gracePeriod=30
Sep 06 10:57:17.460866 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:17.460804    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"
Sep 06 10:57:17.465783 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:17.465760    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6z9cc\" (UniqueName: \"kubernetes.io/projected/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-kube-api-access-6z9cc\") pod \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\" (UID: \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\") "
Sep 06 10:57:17.465926 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:17.465813    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-metrics-certs\") pod \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\" (UID: \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\") "
Sep 06 10:57:17.465926 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:17.465845    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-openshift-service-ca\") pod \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\" (UID: \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\") "
Sep 06 10:57:17.465926 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:17.465874    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-maas-api-tls\") pod \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\" (UID: \"a98f8662-1af8-4697-94ea-ff5a3a4f8a57\") "
Sep 06 10:57:17.466287 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:17.466259    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a98f8662-1af8-4697-94ea-ff5a3a4f8a57" (UID: "a98f8662-1af8-4697-94ea-ff5a3a4f8a57"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:57:17.467956 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:17.467921    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-kube-api-access-6z9cc" (OuterVolumeSpecName: "kube-api-access-6z9cc") pod "a98f8662-1af8-4697-94ea-ff5a3a4f8a57" (UID: "a98f8662-1af8-4697-94ea-ff5a3a4f8a57"). InnerVolumeSpecName "kube-api-access-6z9cc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:57:17.468055 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:17.468007    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a98f8662-1af8-4697-94ea-ff5a3a4f8a57" (UID: "a98f8662-1af8-4697-94ea-ff5a3a4f8a57"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 10:57:17.468268 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:17.468240    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a98f8662-1af8-4697-94ea-ff5a3a4f8a57" (UID: "a98f8662-1af8-4697-94ea-ff5a3a4f8a57"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 10:57:17.567159 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:17.567127    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z9cc\" (UniqueName: \"kubernetes.io/projected/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-kube-api-access-6z9cc\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:17.567159 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:17.567155    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-metrics-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:17.567159 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:17.567165    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-openshift-service-ca\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:17.567373 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:17.567174    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a98f8662-1af8-4697-94ea-ff5a3a4f8a57-maas-api-tls\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:18.281824 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:18.281792    2557 generic.go:356] "Generic (PLEG): container finished" podID="a98f8662-1af8-4697-94ea-ff5a3a4f8a57" containerID="d72a6377290e4c820d13fc7b9430d8a57b9293fd5bf8e136bceb2f39686335ab" exitCode=0
Sep 06 10:57:18.282254 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:18.281835    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"
Sep 06 10:57:18.282254 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:18.281873    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c" event={"ID":"a98f8662-1af8-4697-94ea-ff5a3a4f8a57","Type":"ContainerDied","Data":"d72a6377290e4c820d13fc7b9430d8a57b9293fd5bf8e136bceb2f39686335ab"}
Sep 06 10:57:18.282254 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:18.281916    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c" event={"ID":"a98f8662-1af8-4697-94ea-ff5a3a4f8a57","Type":"ContainerDied","Data":"538b9893f0a78ea9600af6e6174739582850478c5f3fb13ddbdae39ca612a0c5"}
Sep 06 10:57:18.282254 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:18.281938    2557 scope.go:117] "RemoveContainer" containerID="d72a6377290e4c820d13fc7b9430d8a57b9293fd5bf8e136bceb2f39686335ab"
Sep 06 10:57:18.290771 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:18.290754    2557 scope.go:117] "RemoveContainer" containerID="d72a6377290e4c820d13fc7b9430d8a57b9293fd5bf8e136bceb2f39686335ab"
Sep 06 10:57:18.291027 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:57:18.291011    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d72a6377290e4c820d13fc7b9430d8a57b9293fd5bf8e136bceb2f39686335ab\": container with ID starting with d72a6377290e4c820d13fc7b9430d8a57b9293fd5bf8e136bceb2f39686335ab not found: ID does not exist" containerID="d72a6377290e4c820d13fc7b9430d8a57b9293fd5bf8e136bceb2f39686335ab"
Sep 06 10:57:18.291077 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:18.291035    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d72a6377290e4c820d13fc7b9430d8a57b9293fd5bf8e136bceb2f39686335ab"} err="failed to get container status \"d72a6377290e4c820d13fc7b9430d8a57b9293fd5bf8e136bceb2f39686335ab\": rpc error: code = NotFound desc = could not find container \"d72a6377290e4c820d13fc7b9430d8a57b9293fd5bf8e136bceb2f39686335ab\": container with ID starting with d72a6377290e4c820d13fc7b9430d8a57b9293fd5bf8e136bceb2f39686335ab not found: ID does not exist"
Sep 06 10:57:18.301008 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:18.300955    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"]
Sep 06 10:57:18.306863 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:18.306843    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c14da-6df5655647-9nc9c"]
Sep 06 10:57:20.043699 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:20.043666    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a98f8662-1af8-4697-94ea-ff5a3a4f8a57" path="/var/lib/kubelet/pods/a98f8662-1af8-4697-94ea-ff5a3a4f8a57/volumes"
Sep 06 10:57:21.113185 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:21.113150    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:57:23.877237 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:23.877195    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"]
Sep 06 10:57:23.877764 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:23.877448    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh" podUID="0faaade3-425c-4452-a3ea-a23a7219e8ae" containerName="istio-proxy" containerID="cri-o://5c0d73fc15e5ae8cb4ed9b1a24b0e286c1b41250018b5b34c865aa465eed5da3" gracePeriod=30
Sep 06 10:57:24.239249 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.239164    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl"]
Sep 06 10:57:24.239577 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.239563    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31d11217-4f66-4a6e-99f4-aed0cdcbb87f" containerName="maas-api"
Sep 06 10:57:24.239577 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.239578    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="31d11217-4f66-4a6e-99f4-aed0cdcbb87f" containerName="maas-api"
Sep 06 10:57:24.239677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.239587    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a98f8662-1af8-4697-94ea-ff5a3a4f8a57" containerName="maas-api"
Sep 06 10:57:24.239677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.239592    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a98f8662-1af8-4697-94ea-ff5a3a4f8a57" containerName="maas-api"
Sep 06 10:57:24.239677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.239605    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dc570f5-e559-48cc-8dcd-95fcae8ee57f" containerName="istio-proxy"
Sep 06 10:57:24.239677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.239610    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dc570f5-e559-48cc-8dcd-95fcae8ee57f" containerName="istio-proxy"
Sep 06 10:57:24.239677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.239617    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" containerName="istio-proxy"
Sep 06 10:57:24.239677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.239623    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" containerName="istio-proxy"
Sep 06 10:57:24.239677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.239631    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" containerName="istio-proxy"
Sep 06 10:57:24.239677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.239636    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" containerName="istio-proxy"
Sep 06 10:57:24.239677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.239644    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dc570f5-e559-48cc-8dcd-95fcae8ee57f" containerName="istio-proxy"
Sep 06 10:57:24.239677 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.239650    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dc570f5-e559-48cc-8dcd-95fcae8ee57f" containerName="istio-proxy"
Sep 06 10:57:24.240018 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.239715    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dc570f5-e559-48cc-8dcd-95fcae8ee57f" containerName="istio-proxy"
Sep 06 10:57:24.240018 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.239723    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" containerName="istio-proxy"
Sep 06 10:57:24.240018 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.239729    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e44fe83-c490-4c3f-bfd8-b4821bf34fcd" containerName="istio-proxy"
Sep 06 10:57:24.240018 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.239736    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="31d11217-4f66-4a6e-99f4-aed0cdcbb87f" containerName="maas-api"
Sep 06 10:57:24.240018 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.239742    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a98f8662-1af8-4697-94ea-ff5a3a4f8a57" containerName="maas-api"
Sep 06 10:57:24.243143 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.243119    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl"
Sep 06 10:57:24.245316 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.245297    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\""
Sep 06 10:57:24.245441 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.245334    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 10:57:24.249342 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.249314    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl"]
Sep 06 10:57:24.326228 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.326190    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d801f3d-c785-48d5-a05a-8ff208583bac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl\" (UID: \"8d801f3d-c785-48d5-a05a-8ff208583bac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl"
Sep 06 10:57:24.427731 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.427696    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d801f3d-c785-48d5-a05a-8ff208583bac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl\" (UID: \"8d801f3d-c785-48d5-a05a-8ff208583bac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl"
Sep 06 10:57:24.428300 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.428279    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d801f3d-c785-48d5-a05a-8ff208583bac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl\" (UID: \"8d801f3d-c785-48d5-a05a-8ff208583bac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl"
Sep 06 10:57:24.555970 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.555886    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl"
Sep 06 10:57:24.829091 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:24.829014    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl"]
Sep 06 10:57:24.832380 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:57:24.832352    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d801f3d_c785_48d5_a05a_8ff208583bac.slice/crio-a39a5e47d2174882beebb7ce3988137e5dc31c68fcf0ba1e2942a944f6489815 WatchSource:0}: Error finding container a39a5e47d2174882beebb7ce3988137e5dc31c68fcf0ba1e2942a944f6489815: Status 404 returned error can't find the container with id a39a5e47d2174882beebb7ce3988137e5dc31c68fcf0ba1e2942a944f6489815
Sep 06 10:57:25.320217 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:25.320183    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl" event={"ID":"8d801f3d-c785-48d5-a05a-8ff208583bac","Type":"ContainerStarted","Data":"08ef3e195493b0f304aa4ed3c38ba19d3380e554abc395b10ff5d8c2136ec546"}
Sep 06 10:57:25.320661 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:25.320234    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl" event={"ID":"8d801f3d-c785-48d5-a05a-8ff208583bac","Type":"ContainerStarted","Data":"a39a5e47d2174882beebb7ce3988137e5dc31c68fcf0ba1e2942a944f6489815"}
Sep 06 10:57:25.335452 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:25.335399    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl" podStartSLOduration=1.3353844719999999 podStartE2EDuration="1.335384472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:57:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:57:25.333451047 +0000 UTC m=+1657.783231814" watchObservedRunningTime="2026-09-06 10:57:25.335384472 +0000 UTC m=+1657.785165278"
Sep 06 10:57:26.326794 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:26.326755    2557 generic.go:356] "Generic (PLEG): container finished" podID="8d801f3d-c785-48d5-a05a-8ff208583bac" containerID="08ef3e195493b0f304aa4ed3c38ba19d3380e554abc395b10ff5d8c2136ec546" exitCode=0
Sep 06 10:57:26.327246 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:26.326795    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl" event={"ID":"8d801f3d-c785-48d5-a05a-8ff208583bac","Type":"ContainerDied","Data":"08ef3e195493b0f304aa4ed3c38ba19d3380e554abc395b10ff5d8c2136ec546"}
Sep 06 10:57:27.466781 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:27.466749    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl"
Sep 06 10:57:27.558865 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:27.558836    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d801f3d-c785-48d5-a05a-8ff208583bac-maas-api-service-ca\") pod \"8d801f3d-c785-48d5-a05a-8ff208583bac\" (UID: \"8d801f3d-c785-48d5-a05a-8ff208583bac\") "
Sep 06 10:57:27.559169 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:27.559147    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d801f3d-c785-48d5-a05a-8ff208583bac-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8d801f3d-c785-48d5-a05a-8ff208583bac" (UID: "8d801f3d-c785-48d5-a05a-8ff208583bac"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:57:27.659804 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:27.659760    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d801f3d-c785-48d5-a05a-8ff208583bac-maas-api-service-ca\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:28.338357 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:28.338317    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl" event={"ID":"8d801f3d-c785-48d5-a05a-8ff208583bac","Type":"ContainerDied","Data":"a39a5e47d2174882beebb7ce3988137e5dc31c68fcf0ba1e2942a944f6489815"}
Sep 06 10:57:28.338539 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:28.338357    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl"
Sep 06 10:57:28.338539 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:28.338372    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a39a5e47d2174882beebb7ce3988137e5dc31c68fcf0ba1e2942a944f6489815"
Sep 06 10:57:29.130273 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.130248    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:57:29.174439 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.174363    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-workload-certs\") pod \"0faaade3-425c-4452-a3ea-a23a7219e8ae\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") "
Sep 06 10:57:29.174439 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.174433    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-data\") pod \"0faaade3-425c-4452-a3ea-a23a7219e8ae\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") "
Sep 06 10:57:29.174685 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.174474    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-podinfo\") pod \"0faaade3-425c-4452-a3ea-a23a7219e8ae\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") "
Sep 06 10:57:29.174685 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.174518    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-token\") pod \"0faaade3-425c-4452-a3ea-a23a7219e8ae\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") "
Sep 06 10:57:29.174685 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.174580    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-workload-socket\") pod \"0faaade3-425c-4452-a3ea-a23a7219e8ae\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") "
Sep 06 10:57:29.174685 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.174612    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-envoy\") pod \"0faaade3-425c-4452-a3ea-a23a7219e8ae\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") "
Sep 06 10:57:29.174685 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.174652    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-credential-socket\") pod \"0faaade3-425c-4452-a3ea-a23a7219e8ae\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") "
Sep 06 10:57:29.174951 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.174686    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0faaade3-425c-4452-a3ea-a23a7219e8ae" (UID: "0faaade3-425c-4452-a3ea-a23a7219e8ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:57:29.174951 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.174700    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrh8s\" (UniqueName: \"kubernetes.io/projected/0faaade3-425c-4452-a3ea-a23a7219e8ae-kube-api-access-lrh8s\") pod \"0faaade3-425c-4452-a3ea-a23a7219e8ae\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") "
Sep 06 10:57:29.174951 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.174723    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0faaade3-425c-4452-a3ea-a23a7219e8ae-istiod-ca-cert\") pod \"0faaade3-425c-4452-a3ea-a23a7219e8ae\" (UID: \"0faaade3-425c-4452-a3ea-a23a7219e8ae\") "
Sep 06 10:57:29.174951 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.174784    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-data" (OuterVolumeSpecName: "istio-data") pod "0faaade3-425c-4452-a3ea-a23a7219e8ae" (UID: "0faaade3-425c-4452-a3ea-a23a7219e8ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:57:29.175153 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.175017    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-workload-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:29.175153 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.175035    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-data\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:29.175153 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.175062    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0faaade3-425c-4452-a3ea-a23a7219e8ae" (UID: "0faaade3-425c-4452-a3ea-a23a7219e8ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:57:29.175320 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.175243    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0faaade3-425c-4452-a3ea-a23a7219e8ae" (UID: "0faaade3-425c-4452-a3ea-a23a7219e8ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:57:29.175320 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.175311    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0faaade3-425c-4452-a3ea-a23a7219e8ae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0faaade3-425c-4452-a3ea-a23a7219e8ae" (UID: "0faaade3-425c-4452-a3ea-a23a7219e8ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:57:29.176899 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.176865    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0faaade3-425c-4452-a3ea-a23a7219e8ae" (UID: "0faaade3-425c-4452-a3ea-a23a7219e8ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 10:57:29.177594 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.177568    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-token" (OuterVolumeSpecName: "istio-token") pod "0faaade3-425c-4452-a3ea-a23a7219e8ae" (UID: "0faaade3-425c-4452-a3ea-a23a7219e8ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:57:29.177687 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.177606    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0faaade3-425c-4452-a3ea-a23a7219e8ae" (UID: "0faaade3-425c-4452-a3ea-a23a7219e8ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:57:29.177687 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.177622    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0faaade3-425c-4452-a3ea-a23a7219e8ae-kube-api-access-lrh8s" (OuterVolumeSpecName: "kube-api-access-lrh8s") pod "0faaade3-425c-4452-a3ea-a23a7219e8ae" (UID: "0faaade3-425c-4452-a3ea-a23a7219e8ae"). InnerVolumeSpecName "kube-api-access-lrh8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:57:29.276299 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.276264    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-workload-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:29.276299 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.276297    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-envoy\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:29.276552 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.276311    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0faaade3-425c-4452-a3ea-a23a7219e8ae-credential-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:29.276552 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.276322    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrh8s\" (UniqueName: \"kubernetes.io/projected/0faaade3-425c-4452-a3ea-a23a7219e8ae-kube-api-access-lrh8s\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:29.276552 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.276334    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0faaade3-425c-4452-a3ea-a23a7219e8ae-istiod-ca-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:29.276552 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.276345    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-podinfo\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:29.276552 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.276355    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0faaade3-425c-4452-a3ea-a23a7219e8ae-istio-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:29.345447 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.345414    2557 generic.go:356] "Generic (PLEG): container finished" podID="0faaade3-425c-4452-a3ea-a23a7219e8ae" containerID="5c0d73fc15e5ae8cb4ed9b1a24b0e286c1b41250018b5b34c865aa465eed5da3" exitCode=0
Sep 06 10:57:29.345644 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.345463    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"
Sep 06 10:57:29.345644 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.345482    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh" event={"ID":"0faaade3-425c-4452-a3ea-a23a7219e8ae","Type":"ContainerDied","Data":"5c0d73fc15e5ae8cb4ed9b1a24b0e286c1b41250018b5b34c865aa465eed5da3"}
Sep 06 10:57:29.345644 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.345554    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh" event={"ID":"0faaade3-425c-4452-a3ea-a23a7219e8ae","Type":"ContainerDied","Data":"5c5c15cf7867a3accefbda7fcfeecbbe47068c22405c0b88894f2ca45940f924"}
Sep 06 10:57:29.345644 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.345576    2557 scope.go:117] "RemoveContainer" containerID="5c0d73fc15e5ae8cb4ed9b1a24b0e286c1b41250018b5b34c865aa465eed5da3"
Sep 06 10:57:29.354871 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.354852    2557 scope.go:117] "RemoveContainer" containerID="372431e7cbfe7ef0e679449918c44523538c73a137070734a9294a8b30ccd362"
Sep 06 10:57:29.362911 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.362873    2557 scope.go:117] "RemoveContainer" containerID="5c0d73fc15e5ae8cb4ed9b1a24b0e286c1b41250018b5b34c865aa465eed5da3"
Sep 06 10:57:29.363204 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:57:29.363182    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c0d73fc15e5ae8cb4ed9b1a24b0e286c1b41250018b5b34c865aa465eed5da3\": container with ID starting with 5c0d73fc15e5ae8cb4ed9b1a24b0e286c1b41250018b5b34c865aa465eed5da3 not found: ID does not exist" containerID="5c0d73fc15e5ae8cb4ed9b1a24b0e286c1b41250018b5b34c865aa465eed5da3"
Sep 06 10:57:29.363301 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.363211    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c0d73fc15e5ae8cb4ed9b1a24b0e286c1b41250018b5b34c865aa465eed5da3"} err="failed to get container status \"5c0d73fc15e5ae8cb4ed9b1a24b0e286c1b41250018b5b34c865aa465eed5da3\": rpc error: code = NotFound desc = could not find container \"5c0d73fc15e5ae8cb4ed9b1a24b0e286c1b41250018b5b34c865aa465eed5da3\": container with ID starting with 5c0d73fc15e5ae8cb4ed9b1a24b0e286c1b41250018b5b34c865aa465eed5da3 not found: ID does not exist"
Sep 06 10:57:29.363301 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.363234    2557 scope.go:117] "RemoveContainer" containerID="372431e7cbfe7ef0e679449918c44523538c73a137070734a9294a8b30ccd362"
Sep 06 10:57:29.363532 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:57:29.363516    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"372431e7cbfe7ef0e679449918c44523538c73a137070734a9294a8b30ccd362\": container with ID starting with 372431e7cbfe7ef0e679449918c44523538c73a137070734a9294a8b30ccd362 not found: ID does not exist" containerID="372431e7cbfe7ef0e679449918c44523538c73a137070734a9294a8b30ccd362"
Sep 06 10:57:29.363615 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.363535    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"372431e7cbfe7ef0e679449918c44523538c73a137070734a9294a8b30ccd362"} err="failed to get container status \"372431e7cbfe7ef0e679449918c44523538c73a137070734a9294a8b30ccd362\": rpc error: code = NotFound desc = could not find container \"372431e7cbfe7ef0e679449918c44523538c73a137070734a9294a8b30ccd362\": container with ID starting with 372431e7cbfe7ef0e679449918c44523538c73a137070734a9294a8b30ccd362 not found: ID does not exist"
Sep 06 10:57:29.366141 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.366121    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"]
Sep 06 10:57:29.370115 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:29.370095    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-8c14da-openshift-default-76cc4c569d-6s8vh"]
Sep 06 10:57:30.036620 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:30.036583    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0faaade3-425c-4452-a3ea-a23a7219e8ae" path="/var/lib/kubelet/pods/0faaade3-425c-4452-a3ea-a23a7219e8ae/volumes"
Sep 06 10:57:34.830795 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:34.830758    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl"]
Sep 06 10:57:34.832675 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:34.832640    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2eb161-ztgcl"]
Sep 06 10:57:36.036826 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:36.036791    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d801f3d-c785-48d5-a05a-8ff208583bac" path="/var/lib/kubelet/pods/8d801f3d-c785-48d5-a05a-8ff208583bac/volumes"
Sep 06 10:57:38.214510 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:38.211709    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:57:41.691212 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:41.691174    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-eb4b45-khjtm"]
Sep 06 10:57:41.691606 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:41.691546    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0faaade3-425c-4452-a3ea-a23a7219e8ae" containerName="istio-proxy"
Sep 06 10:57:41.691606 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:41.691558    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0faaade3-425c-4452-a3ea-a23a7219e8ae" containerName="istio-proxy"
Sep 06 10:57:41.691606 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:41.691583    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0faaade3-425c-4452-a3ea-a23a7219e8ae" containerName="istio-proxy"
Sep 06 10:57:41.691606 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:41.691588    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0faaade3-425c-4452-a3ea-a23a7219e8ae" containerName="istio-proxy"
Sep 06 10:57:41.691606 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:41.691597    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d801f3d-c785-48d5-a05a-8ff208583bac" containerName="revoke-keys"
Sep 06 10:57:41.691606 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:41.691603    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d801f3d-c785-48d5-a05a-8ff208583bac" containerName="revoke-keys"
Sep 06 10:57:41.691802 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:41.691658    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dc570f5-e559-48cc-8dcd-95fcae8ee57f" containerName="istio-proxy"
Sep 06 10:57:41.691802 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:41.691666    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0faaade3-425c-4452-a3ea-a23a7219e8ae" containerName="istio-proxy"
Sep 06 10:57:41.691802 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:41.691687    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d801f3d-c785-48d5-a05a-8ff208583bac" containerName="revoke-keys"
Sep 06 10:57:41.691802 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:41.691695    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0faaade3-425c-4452-a3ea-a23a7219e8ae" containerName="istio-proxy"
Sep 06 10:57:41.696426 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:41.696409    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-eb4b45-khjtm"
Sep 06 10:57:41.699133 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:41.699099    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\""
Sep 06 10:57:41.699350 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:41.699330    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 10:57:41.700709 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:41.700691    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-eb4b45-khjtm"]
Sep 06 10:57:41.790308 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:41.790279    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5388bcf3-4402-406a-9c34-636dc858dfb3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-eb4b45-khjtm\" (UID: \"5388bcf3-4402-406a-9c34-636dc858dfb3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-eb4b45-khjtm"
Sep 06 10:57:41.890773 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:41.890745    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5388bcf3-4402-406a-9c34-636dc858dfb3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-eb4b45-khjtm\" (UID: \"5388bcf3-4402-406a-9c34-636dc858dfb3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-eb4b45-khjtm"
Sep 06 10:57:41.891294 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:41.891275    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5388bcf3-4402-406a-9c34-636dc858dfb3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-eb4b45-khjtm\" (UID: \"5388bcf3-4402-406a-9c34-636dc858dfb3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-eb4b45-khjtm"
Sep 06 10:57:42.008728 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:42.008658    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-eb4b45-khjtm"
Sep 06 10:57:42.276172 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:42.276149    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-eb4b45-khjtm"]
Sep 06 10:57:42.278194 ip-10-0-133-153 kubenswrapper[2557]: W0906 10:57:42.278159    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5388bcf3_4402_406a_9c34_636dc858dfb3.slice/crio-0779045c3c9a93b019bde3a2e8cd51b5ad34a1d5cfc716d795ca0f289eea9ec5 WatchSource:0}: Error finding container 0779045c3c9a93b019bde3a2e8cd51b5ad34a1d5cfc716d795ca0f289eea9ec5: Status 404 returned error can't find the container with id 0779045c3c9a93b019bde3a2e8cd51b5ad34a1d5cfc716d795ca0f289eea9ec5
Sep 06 10:57:42.415130 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:42.415094    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-eb4b45-khjtm" event={"ID":"5388bcf3-4402-406a-9c34-636dc858dfb3","Type":"ContainerStarted","Data":"0779045c3c9a93b019bde3a2e8cd51b5ad34a1d5cfc716d795ca0f289eea9ec5"}
Sep 06 10:57:43.421966 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:43.421935    2557 generic.go:356] "Generic (PLEG): container finished" podID="5388bcf3-4402-406a-9c34-636dc858dfb3" containerID="e38ca8ef4eef1462da250846397cd8de68d08621f5e7e550ca81c8dfaedea2e9" exitCode=0
Sep 06 10:57:43.422347 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:43.422015    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-eb4b45-khjtm" event={"ID":"5388bcf3-4402-406a-9c34-636dc858dfb3","Type":"ContainerDied","Data":"e38ca8ef4eef1462da250846397cd8de68d08621f5e7e550ca81c8dfaedea2e9"}
Sep 06 10:57:44.559977 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:44.559948    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-eb4b45-khjtm"
Sep 06 10:57:44.719350 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:44.719273    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5388bcf3-4402-406a-9c34-636dc858dfb3-maas-api-service-ca\") pod \"5388bcf3-4402-406a-9c34-636dc858dfb3\" (UID: \"5388bcf3-4402-406a-9c34-636dc858dfb3\") "
Sep 06 10:57:44.719655 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:44.719630    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5388bcf3-4402-406a-9c34-636dc858dfb3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5388bcf3-4402-406a-9c34-636dc858dfb3" (UID: "5388bcf3-4402-406a-9c34-636dc858dfb3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:57:44.820073 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:44.820041    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5388bcf3-4402-406a-9c34-636dc858dfb3-maas-api-service-ca\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:57:45.433119 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:45.433088    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-eb4b45-khjtm"
Sep 06 10:57:45.433119 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:45.433113    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-eb4b45-khjtm" event={"ID":"5388bcf3-4402-406a-9c34-636dc858dfb3","Type":"ContainerDied","Data":"0779045c3c9a93b019bde3a2e8cd51b5ad34a1d5cfc716d795ca0f289eea9ec5"}
Sep 06 10:57:45.433319 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:45.433157    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0779045c3c9a93b019bde3a2e8cd51b5ad34a1d5cfc716d795ca0f289eea9ec5"
Sep 06 10:57:52.415975 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:52.415941    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-eb4b45-khjtm"]
Sep 06 10:57:52.419473 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:52.419445    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-eb4b45-khjtm"]
Sep 06 10:57:54.037540 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:54.037507    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5388bcf3-4402-406a-9c34-636dc858dfb3" path="/var/lib/kubelet/pods/5388bcf3-4402-406a-9c34-636dc858dfb3/volumes"
Sep 06 10:57:55.220965 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:55.220931    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:57:58.995091 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:58.995063    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"]
Sep 06 10:57:58.995520 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:58.995318    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs" podUID="dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" containerName="istio-proxy" containerID="cri-o://d77b15da80019fd6a04e3dd11e7dff7febb12d99c0e0ed8d618f7519e0bf6e66" gracePeriod=30
Sep 06 10:57:59.696253 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:59.696217    2557 patch_prober.go:28] interesting pod/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 06 10:57:59.696456 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:57:59.696314    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs" podUID="dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 10:58:04.264900 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.264840    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:58:04.288386 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.288358    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-envoy\") pod \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") "
Sep 06 10:58:04.288566 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.288404    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-workload-certs\") pod \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") "
Sep 06 10:58:04.288566 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.288454    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-data\") pod \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") "
Sep 06 10:58:04.288566 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.288505    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-workload-socket\") pod \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") "
Sep 06 10:58:04.288566 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.288540    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istiod-ca-cert\") pod \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") "
Sep 06 10:58:04.288566 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.288561    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-token\") pod \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") "
Sep 06 10:58:04.288833 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.288585    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-credential-socket\") pod \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") "
Sep 06 10:58:04.288833 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.288646    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-podinfo\") pod \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") "
Sep 06 10:58:04.288833 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.288700    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2swtt\" (UniqueName: \"kubernetes.io/projected/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-kube-api-access-2swtt\") pod \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\" (UID: \"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c\") "
Sep 06 10:58:04.288833 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.288705    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" (UID: "dd66f3fa-b46c-4850-9f62-c97c1a9fa32c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:58:04.288833 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.288771    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-data" (OuterVolumeSpecName: "istio-data") pod "dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" (UID: "dd66f3fa-b46c-4850-9f62-c97c1a9fa32c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:58:04.289190 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.288868    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" (UID: "dd66f3fa-b46c-4850-9f62-c97c1a9fa32c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:58:04.289190 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.288956    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" (UID: "dd66f3fa-b46c-4850-9f62-c97c1a9fa32c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 10:58:04.289190 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.289106    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" (UID: "dd66f3fa-b46c-4850-9f62-c97c1a9fa32c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:58:04.289190 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.289135    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-workload-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:58:04.289190 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.289154    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-data\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:58:04.289190 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.289167    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-workload-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:58:04.289190 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.289180    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istiod-ca-cert\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:58:04.291449 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.291412    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-token" (OuterVolumeSpecName: "istio-token") pod "dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" (UID: "dd66f3fa-b46c-4850-9f62-c97c1a9fa32c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:58:04.291601 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.291433    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-kube-api-access-2swtt" (OuterVolumeSpecName: "kube-api-access-2swtt") pod "dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" (UID: "dd66f3fa-b46c-4850-9f62-c97c1a9fa32c"). InnerVolumeSpecName "kube-api-access-2swtt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 10:58:04.291693 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.291662    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" (UID: "dd66f3fa-b46c-4850-9f62-c97c1a9fa32c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 10:58:04.291866 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.291838    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" (UID: "dd66f3fa-b46c-4850-9f62-c97c1a9fa32c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 10:58:04.389739 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.389712    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-token\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:58:04.389739 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.389734    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-credential-socket\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:58:04.389936 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.389745    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-podinfo\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:58:04.389936 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.389754    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2swtt\" (UniqueName: \"kubernetes.io/projected/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-kube-api-access-2swtt\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:58:04.389936 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.389763    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c-istio-envoy\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 10:58:04.540425 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.540341    2557 generic.go:356] "Generic (PLEG): container finished" podID="dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" containerID="d77b15da80019fd6a04e3dd11e7dff7febb12d99c0e0ed8d618f7519e0bf6e66" exitCode=0
Sep 06 10:58:04.540425 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.540404    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"
Sep 06 10:58:04.540632 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.540434    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs" event={"ID":"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c","Type":"ContainerDied","Data":"d77b15da80019fd6a04e3dd11e7dff7febb12d99c0e0ed8d618f7519e0bf6e66"}
Sep 06 10:58:04.540632 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.540531    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs" event={"ID":"dd66f3fa-b46c-4850-9f62-c97c1a9fa32c","Type":"ContainerDied","Data":"11ce0f97b2974f443a21e4cc85bedb97e2be7a5072bef83732542e548b019b23"}
Sep 06 10:58:04.540632 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.540556    2557 scope.go:117] "RemoveContainer" containerID="d77b15da80019fd6a04e3dd11e7dff7febb12d99c0e0ed8d618f7519e0bf6e66"
Sep 06 10:58:04.550068 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.550050    2557 scope.go:117] "RemoveContainer" containerID="5cf684e5812f4beb7e98d8266f5641b0ef1ac6d08fa7d482b9a4e9e83f7ec762"
Sep 06 10:58:04.557884 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.557865    2557 scope.go:117] "RemoveContainer" containerID="d77b15da80019fd6a04e3dd11e7dff7febb12d99c0e0ed8d618f7519e0bf6e66"
Sep 06 10:58:04.558235 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:58:04.558205    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d77b15da80019fd6a04e3dd11e7dff7febb12d99c0e0ed8d618f7519e0bf6e66\": container with ID starting with d77b15da80019fd6a04e3dd11e7dff7febb12d99c0e0ed8d618f7519e0bf6e66 not found: ID does not exist" containerID="d77b15da80019fd6a04e3dd11e7dff7febb12d99c0e0ed8d618f7519e0bf6e66"
Sep 06 10:58:04.558365 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.558245    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d77b15da80019fd6a04e3dd11e7dff7febb12d99c0e0ed8d618f7519e0bf6e66"} err="failed to get container status \"d77b15da80019fd6a04e3dd11e7dff7febb12d99c0e0ed8d618f7519e0bf6e66\": rpc error: code = NotFound desc = could not find container \"d77b15da80019fd6a04e3dd11e7dff7febb12d99c0e0ed8d618f7519e0bf6e66\": container with ID starting with d77b15da80019fd6a04e3dd11e7dff7febb12d99c0e0ed8d618f7519e0bf6e66 not found: ID does not exist"
Sep 06 10:58:04.558365 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.558269    2557 scope.go:117] "RemoveContainer" containerID="5cf684e5812f4beb7e98d8266f5641b0ef1ac6d08fa7d482b9a4e9e83f7ec762"
Sep 06 10:58:04.558696 ip-10-0-133-153 kubenswrapper[2557]: E0906 10:58:04.558635    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cf684e5812f4beb7e98d8266f5641b0ef1ac6d08fa7d482b9a4e9e83f7ec762\": container with ID starting with 5cf684e5812f4beb7e98d8266f5641b0ef1ac6d08fa7d482b9a4e9e83f7ec762 not found: ID does not exist" containerID="5cf684e5812f4beb7e98d8266f5641b0ef1ac6d08fa7d482b9a4e9e83f7ec762"
Sep 06 10:58:04.558696 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.558666    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cf684e5812f4beb7e98d8266f5641b0ef1ac6d08fa7d482b9a4e9e83f7ec762"} err="failed to get container status \"5cf684e5812f4beb7e98d8266f5641b0ef1ac6d08fa7d482b9a4e9e83f7ec762\": rpc error: code = NotFound desc = could not find container \"5cf684e5812f4beb7e98d8266f5641b0ef1ac6d08fa7d482b9a4e9e83f7ec762\": container with ID starting with 5cf684e5812f4beb7e98d8266f5641b0ef1ac6d08fa7d482b9a4e9e83f7ec762 not found: ID does not exist"
Sep 06 10:58:04.560581 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.560563    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"]
Sep 06 10:58:04.565660 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:04.565640    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-eb4b45-openshift-default-7d8d65f8cd-5nqvs"]
Sep 06 10:58:05.310794 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:05.310760    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:58:06.037545 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:06.037509    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" path="/var/lib/kubelet/pods/dd66f3fa-b46c-4850-9f62-c97c1a9fa32c/volumes"
Sep 06 10:58:48.526148 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:58:48.526118    2557 scope.go:117] "RemoveContainer" containerID="a267b0d3e66130fa9b7a016f4f4691d5f0f2bfd799af16afd88335155c27c34a"
Sep 06 10:59:23.611004 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:59:23.610967    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:59:48.157690 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:59:48.157659    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/2.log"
Sep 06 10:59:48.168736 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:59:48.168710    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/2.log"
Sep 06 10:59:48.549804 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:59:48.549721    2557 scope.go:117] "RemoveContainer" containerID="676fecd529c19d08e39c36f80053909ae799dc0b8d3fd4c3c7dd597595c9aa1a"
Sep 06 10:59:48.615992 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:59:48.615955    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 10:59:55.608206 ip-10-0-133-153 kubenswrapper[2557]: I0906 10:59:55.608169    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 11:00:05.909783 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:05.909737    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 11:00:23.817372 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:23.817333    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 11:00:31.914663 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:31.914620    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 11:00:41.430888 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:41.430850    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp"]
Sep 06 11:00:41.431313 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:41.431110    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp" podUID="91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1" containerName="manager" containerID="cri-o://c9b6c3fcc66f73208c69fec083966b7e1bcd376fbc640e6c93a4131af5406eee" gracePeriod=10
Sep 06 11:00:41.783464 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:41.783435    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp"
Sep 06 11:00:41.946135 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:41.946092    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfd6q\" (UniqueName: \"kubernetes.io/projected/91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1-kube-api-access-dfd6q\") pod \"91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1\" (UID: \"91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1\") "
Sep 06 11:00:41.946319 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:41.946167    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1-extensions-socket-volume\") pod \"91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1\" (UID: \"91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1\") "
Sep 06 11:00:41.946565 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:41.946537    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1" (UID: "91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 11:00:41.948255 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:41.948227    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1-kube-api-access-dfd6q" (OuterVolumeSpecName: "kube-api-access-dfd6q") pod "91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1" (UID: "91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1"). InnerVolumeSpecName "kube-api-access-dfd6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 11:00:42.046913 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:42.046888    2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1-extensions-socket-volume\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 11:00:42.046913 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:42.046912    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfd6q\" (UniqueName: \"kubernetes.io/projected/91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1-kube-api-access-dfd6q\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 11:00:42.351929 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:42.351893    2557 generic.go:356] "Generic (PLEG): container finished" podID="91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1" containerID="c9b6c3fcc66f73208c69fec083966b7e1bcd376fbc640e6c93a4131af5406eee" exitCode=0
Sep 06 11:00:42.352131 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:42.351941    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp"
Sep 06 11:00:42.352131 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:42.351985    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp" event={"ID":"91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1","Type":"ContainerDied","Data":"c9b6c3fcc66f73208c69fec083966b7e1bcd376fbc640e6c93a4131af5406eee"}
Sep 06 11:00:42.352131 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:42.352049    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp" event={"ID":"91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1","Type":"ContainerDied","Data":"51afe812bea64803ef6c434cebebc3490d8d4796015ecd208ecf89b1b1fea238"}
Sep 06 11:00:42.352131 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:42.352079    2557 scope.go:117] "RemoveContainer" containerID="c9b6c3fcc66f73208c69fec083966b7e1bcd376fbc640e6c93a4131af5406eee"
Sep 06 11:00:42.361788 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:42.361769    2557 scope.go:117] "RemoveContainer" containerID="c9b6c3fcc66f73208c69fec083966b7e1bcd376fbc640e6c93a4131af5406eee"
Sep 06 11:00:42.362037 ip-10-0-133-153 kubenswrapper[2557]: E0906 11:00:42.362019    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9b6c3fcc66f73208c69fec083966b7e1bcd376fbc640e6c93a4131af5406eee\": container with ID starting with c9b6c3fcc66f73208c69fec083966b7e1bcd376fbc640e6c93a4131af5406eee not found: ID does not exist" containerID="c9b6c3fcc66f73208c69fec083966b7e1bcd376fbc640e6c93a4131af5406eee"
Sep 06 11:00:42.362091 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:42.362043    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9b6c3fcc66f73208c69fec083966b7e1bcd376fbc640e6c93a4131af5406eee"} err="failed to get container status \"c9b6c3fcc66f73208c69fec083966b7e1bcd376fbc640e6c93a4131af5406eee\": rpc error: code = NotFound desc = could not find container \"c9b6c3fcc66f73208c69fec083966b7e1bcd376fbc640e6c93a4131af5406eee\": container with ID starting with c9b6c3fcc66f73208c69fec083966b7e1bcd376fbc640e6c93a4131af5406eee not found: ID does not exist"
Sep 06 11:00:42.369342 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:42.369320    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp"]
Sep 06 11:00:42.372365 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:42.372335    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rj4tp"]
Sep 06 11:00:44.037057 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:00:44.037020    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1" path="/var/lib/kubelet/pods/91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1/volumes"
Sep 06 11:01:42.106253 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.106211    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-xsj2q"]
Sep 06 11:01:42.106840 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.106778    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" containerName="istio-proxy"
Sep 06 11:01:42.106840 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.106799    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" containerName="istio-proxy"
Sep 06 11:01:42.106840 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.106814    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5388bcf3-4402-406a-9c34-636dc858dfb3" containerName="revoke-keys"
Sep 06 11:01:42.106840 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.106823    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5388bcf3-4402-406a-9c34-636dc858dfb3" containerName="revoke-keys"
Sep 06 11:01:42.107131 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.106846    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1" containerName="manager"
Sep 06 11:01:42.107131 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.106855    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1" containerName="manager"
Sep 06 11:01:42.107131 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.106955    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" containerName="istio-proxy"
Sep 06 11:01:42.107131 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.106968    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5388bcf3-4402-406a-9c34-636dc858dfb3" containerName="revoke-keys"
Sep 06 11:01:42.107131 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.106981    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" containerName="istio-proxy"
Sep 06 11:01:42.107131 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.106993    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="91e80eef-0905-4ab9-bf55-bc1ddaf7f8c1" containerName="manager"
Sep 06 11:01:42.110082 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.110059    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-xsj2q"
Sep 06 11:01:42.112462 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.112441    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8lm44\""
Sep 06 11:01:42.121888 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.121865    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-xsj2q"]
Sep 06 11:01:42.261377 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.261339    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f028ca6-f40a-4374-84ae-59315b311745-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-xsj2q\" (UID: \"2f028ca6-f40a-4374-84ae-59315b311745\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-xsj2q"
Sep 06 11:01:42.261533 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.261389    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5flw8\" (UniqueName: \"kubernetes.io/projected/2f028ca6-f40a-4374-84ae-59315b311745-kube-api-access-5flw8\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-xsj2q\" (UID: \"2f028ca6-f40a-4374-84ae-59315b311745\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-xsj2q"
Sep 06 11:01:42.362569 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.362476    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f028ca6-f40a-4374-84ae-59315b311745-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-xsj2q\" (UID: \"2f028ca6-f40a-4374-84ae-59315b311745\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-xsj2q"
Sep 06 11:01:42.362569 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.362550    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5flw8\" (UniqueName: \"kubernetes.io/projected/2f028ca6-f40a-4374-84ae-59315b311745-kube-api-access-5flw8\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-xsj2q\" (UID: \"2f028ca6-f40a-4374-84ae-59315b311745\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-xsj2q"
Sep 06 11:01:42.362828 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.362808    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f028ca6-f40a-4374-84ae-59315b311745-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-xsj2q\" (UID: \"2f028ca6-f40a-4374-84ae-59315b311745\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-xsj2q"
Sep 06 11:01:42.370186 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.370159    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5flw8\" (UniqueName: \"kubernetes.io/projected/2f028ca6-f40a-4374-84ae-59315b311745-kube-api-access-5flw8\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-xsj2q\" (UID: \"2f028ca6-f40a-4374-84ae-59315b311745\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-xsj2q"
Sep 06 11:01:42.420086 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.420056    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-xsj2q"
Sep 06 11:01:42.693193 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:42.693165    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-xsj2q"]
Sep 06 11:01:42.695021 ip-10-0-133-153 kubenswrapper[2557]: W0906 11:01:42.694995    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f028ca6_f40a_4374_84ae_59315b311745.slice/crio-023f53bea8f35484912cbbc734db991632af5e2c36d0dcb3e37b98d1dcc9e90d WatchSource:0}: Error finding container 023f53bea8f35484912cbbc734db991632af5e2c36d0dcb3e37b98d1dcc9e90d: Status 404 returned error can't find the container with id 023f53bea8f35484912cbbc734db991632af5e2c36d0dcb3e37b98d1dcc9e90d
Sep 06 11:01:43.662532 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:43.662472    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-xsj2q" event={"ID":"2f028ca6-f40a-4374-84ae-59315b311745","Type":"ContainerStarted","Data":"d52481b0eea53bb975f7f43c53902eb6567c27bd77fbe2de6c429a25535d6eeb"}
Sep 06 11:01:43.662962 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:43.662555    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-xsj2q" event={"ID":"2f028ca6-f40a-4374-84ae-59315b311745","Type":"ContainerStarted","Data":"023f53bea8f35484912cbbc734db991632af5e2c36d0dcb3e37b98d1dcc9e90d"}
Sep 06 11:01:43.662962 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:43.662601    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-xsj2q"
Sep 06 11:01:43.683423 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:43.683379    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-xsj2q" podStartSLOduration=1.683363484 podStartE2EDuration="1.683363484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 11:01:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 11:01:43.680817007 +0000 UTC m=+1916.130597779" watchObservedRunningTime="2026-09-06 11:01:43.683363484 +0000 UTC m=+1916.133144251"
Sep 06 11:01:54.669917 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:01:54.669882    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-xsj2q"
Sep 06 11:02:01.515285 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:02:01.515249    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 11:02:08.313813 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:02:08.313778    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 11:02:17.716582 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:02:17.716551    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 11:02:24.619056 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:02:24.619023    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 11:02:39.714802 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:02:39.714768    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 11:02:48.412440 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:02:48.412369    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 11:02:56.011754 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:02:56.011720    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 11:03:02.544807 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.544774    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"]
Sep 06 11:03:02.545171 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.545145    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" containerName="istio-proxy"
Sep 06 11:03:02.545171 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.545156    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd66f3fa-b46c-4850-9f62-c97c1a9fa32c" containerName="istio-proxy"
Sep 06 11:03:02.548715 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.548698    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"
Sep 06 11:03:02.552107 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.552082    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 06 11:03:02.552230 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.552084    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q972h\""
Sep 06 11:03:02.552281 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.552225    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 06 11:03:02.552327 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.552283    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 11:03:02.552533 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.552518    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 06 11:03:02.558441 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.558416    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"]
Sep 06 11:03:02.633123 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.633087    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqftk\" (UniqueName: \"kubernetes.io/projected/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-kube-api-access-qqftk\") pod \"maas-api-bd6c55749-hxsqh\" (UID: \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\") " pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"
Sep 06 11:03:02.633296 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.633141    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-metrics-certs\") pod \"maas-api-bd6c55749-hxsqh\" (UID: \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\") " pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"
Sep 06 11:03:02.633296 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.633210    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-maas-api-tls\") pod \"maas-api-bd6c55749-hxsqh\" (UID: \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\") " pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"
Sep 06 11:03:02.633296 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.633261    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-openshift-service-ca\") pod \"maas-api-bd6c55749-hxsqh\" (UID: \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\") " pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"
Sep 06 11:03:02.733989 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.733957    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqftk\" (UniqueName: \"kubernetes.io/projected/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-kube-api-access-qqftk\") pod \"maas-api-bd6c55749-hxsqh\" (UID: \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\") " pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"
Sep 06 11:03:02.734167 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.734002    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-metrics-certs\") pod \"maas-api-bd6c55749-hxsqh\" (UID: \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\") " pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"
Sep 06 11:03:02.734167 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.734028    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-maas-api-tls\") pod \"maas-api-bd6c55749-hxsqh\" (UID: \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\") " pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"
Sep 06 11:03:02.734167 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.734051    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-openshift-service-ca\") pod \"maas-api-bd6c55749-hxsqh\" (UID: \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\") " pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"
Sep 06 11:03:02.734839 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.734811    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-openshift-service-ca\") pod \"maas-api-bd6c55749-hxsqh\" (UID: \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\") " pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"
Sep 06 11:03:02.736539 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.736506    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-maas-api-tls\") pod \"maas-api-bd6c55749-hxsqh\" (UID: \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\") " pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"
Sep 06 11:03:02.736645 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.736554    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-metrics-certs\") pod \"maas-api-bd6c55749-hxsqh\" (UID: \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\") " pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"
Sep 06 11:03:02.740858 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.740838    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqftk\" (UniqueName: \"kubernetes.io/projected/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-kube-api-access-qqftk\") pod \"maas-api-bd6c55749-hxsqh\" (UID: \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\") " pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"
Sep 06 11:03:02.860446 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.860409    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"
Sep 06 11:03:02.909392 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:02.909360    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 11:03:03.133075 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:03.133039    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"]
Sep 06 11:03:03.134603 ip-10-0-133-153 kubenswrapper[2557]: W0906 11:03:03.134571    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ed5d2da_6e3d_4547_8ba4_e7397ae9276c.slice/crio-dc39b5d8564df29211e2255221e3e5e553586313f95f1480996b3413c43756f1 WatchSource:0}: Error finding container dc39b5d8564df29211e2255221e3e5e553586313f95f1480996b3413c43756f1: Status 404 returned error can't find the container with id dc39b5d8564df29211e2255221e3e5e553586313f95f1480996b3413c43756f1
Sep 06 11:03:03.135965 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:03.135940    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 11:03:04.064816 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:04.064714    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh" event={"ID":"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c","Type":"ContainerStarted","Data":"8aa11ee69898d349c053a8ccdb9b5b0f6927d63fdb18513102e9a19dd075ebda"}
Sep 06 11:03:04.065187 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:04.064816    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh" event={"ID":"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c","Type":"ContainerStarted","Data":"dc39b5d8564df29211e2255221e3e5e553586313f95f1480996b3413c43756f1"}
Sep 06 11:03:04.065187 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:04.064845    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"
Sep 06 11:03:04.080607 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:04.080554    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh" podStartSLOduration=1.58922623 podStartE2EDuration="2.080539455s" totalImagesPullingTime="491.313225ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 11:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 11:03:04.079505405 +0000 UTC m=+1996.529286174" watchObservedRunningTime="2026-09-06 11:03:04.080539455 +0000 UTC m=+1996.530320227"
Sep 06 11:03:09.854150 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:09.854114    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"]
Sep 06 11:03:09.854668 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:09.854371    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh" podUID="9ed5d2da-6e3d-4547-8ba4-e7397ae9276c" containerName="maas-api" containerID="cri-o://8aa11ee69898d349c053a8ccdb9b5b0f6927d63fdb18513102e9a19dd075ebda" gracePeriod=30
Sep 06 11:03:09.860350 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:09.860327    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"
Sep 06 11:03:09.884103 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:09.884078    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-66b9899c65-krpz5"]
Sep 06 11:03:09.887419 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:09.887401    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66b9899c65-krpz5"
Sep 06 11:03:09.902707 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:09.902667    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-66b9899c65-krpz5"]
Sep 06 11:03:10.007556 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.007516    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba2ff9d9-e91e-4d30-9d19-27a609e42447-maas-api-tls\") pod \"maas-api-66b9899c65-krpz5\" (UID: \"ba2ff9d9-e91e-4d30-9d19-27a609e42447\") " pod="odh-ai-gateway-infra/maas-api-66b9899c65-krpz5"
Sep 06 11:03:10.007747 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.007630    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba2ff9d9-e91e-4d30-9d19-27a609e42447-openshift-service-ca\") pod \"maas-api-66b9899c65-krpz5\" (UID: \"ba2ff9d9-e91e-4d30-9d19-27a609e42447\") " pod="odh-ai-gateway-infra/maas-api-66b9899c65-krpz5"
Sep 06 11:03:10.007747 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.007695    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba2ff9d9-e91e-4d30-9d19-27a609e42447-metrics-certs\") pod \"maas-api-66b9899c65-krpz5\" (UID: \"ba2ff9d9-e91e-4d30-9d19-27a609e42447\") " pod="odh-ai-gateway-infra/maas-api-66b9899c65-krpz5"
Sep 06 11:03:10.007747 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.007735    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7pfw\" (UniqueName: \"kubernetes.io/projected/ba2ff9d9-e91e-4d30-9d19-27a609e42447-kube-api-access-r7pfw\") pod \"maas-api-66b9899c65-krpz5\" (UID: \"ba2ff9d9-e91e-4d30-9d19-27a609e42447\") " pod="odh-ai-gateway-infra/maas-api-66b9899c65-krpz5"
Sep 06 11:03:10.097914 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.097882    2557 generic.go:356] "Generic (PLEG): container finished" podID="9ed5d2da-6e3d-4547-8ba4-e7397ae9276c" containerID="8aa11ee69898d349c053a8ccdb9b5b0f6927d63fdb18513102e9a19dd075ebda" exitCode=0
Sep 06 11:03:10.098073 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.097920    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh" event={"ID":"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c","Type":"ContainerDied","Data":"8aa11ee69898d349c053a8ccdb9b5b0f6927d63fdb18513102e9a19dd075ebda"}
Sep 06 11:03:10.098073 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.097958    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh" event={"ID":"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c","Type":"ContainerDied","Data":"dc39b5d8564df29211e2255221e3e5e553586313f95f1480996b3413c43756f1"}
Sep 06 11:03:10.098073 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.097974    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc39b5d8564df29211e2255221e3e5e553586313f95f1480996b3413c43756f1"
Sep 06 11:03:10.107500 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.107439    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"
Sep 06 11:03:10.108657 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.108640    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba2ff9d9-e91e-4d30-9d19-27a609e42447-openshift-service-ca\") pod \"maas-api-66b9899c65-krpz5\" (UID: \"ba2ff9d9-e91e-4d30-9d19-27a609e42447\") " pod="odh-ai-gateway-infra/maas-api-66b9899c65-krpz5"
Sep 06 11:03:10.108722 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.108698    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba2ff9d9-e91e-4d30-9d19-27a609e42447-metrics-certs\") pod \"maas-api-66b9899c65-krpz5\" (UID: \"ba2ff9d9-e91e-4d30-9d19-27a609e42447\") " pod="odh-ai-gateway-infra/maas-api-66b9899c65-krpz5"
Sep 06 11:03:10.108768 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.108732    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7pfw\" (UniqueName: \"kubernetes.io/projected/ba2ff9d9-e91e-4d30-9d19-27a609e42447-kube-api-access-r7pfw\") pod \"maas-api-66b9899c65-krpz5\" (UID: \"ba2ff9d9-e91e-4d30-9d19-27a609e42447\") " pod="odh-ai-gateway-infra/maas-api-66b9899c65-krpz5"
Sep 06 11:03:10.108822 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.108773    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba2ff9d9-e91e-4d30-9d19-27a609e42447-maas-api-tls\") pod \"maas-api-66b9899c65-krpz5\" (UID: \"ba2ff9d9-e91e-4d30-9d19-27a609e42447\") " pod="odh-ai-gateway-infra/maas-api-66b9899c65-krpz5"
Sep 06 11:03:10.109307 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.109285    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba2ff9d9-e91e-4d30-9d19-27a609e42447-openshift-service-ca\") pod \"maas-api-66b9899c65-krpz5\" (UID: \"ba2ff9d9-e91e-4d30-9d19-27a609e42447\") " pod="odh-ai-gateway-infra/maas-api-66b9899c65-krpz5"
Sep 06 11:03:10.111124 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.111099    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba2ff9d9-e91e-4d30-9d19-27a609e42447-maas-api-tls\") pod \"maas-api-66b9899c65-krpz5\" (UID: \"ba2ff9d9-e91e-4d30-9d19-27a609e42447\") " pod="odh-ai-gateway-infra/maas-api-66b9899c65-krpz5"
Sep 06 11:03:10.111219 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.111134    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba2ff9d9-e91e-4d30-9d19-27a609e42447-metrics-certs\") pod \"maas-api-66b9899c65-krpz5\" (UID: \"ba2ff9d9-e91e-4d30-9d19-27a609e42447\") " pod="odh-ai-gateway-infra/maas-api-66b9899c65-krpz5"
Sep 06 11:03:10.115479 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.115461    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7pfw\" (UniqueName: \"kubernetes.io/projected/ba2ff9d9-e91e-4d30-9d19-27a609e42447-kube-api-access-r7pfw\") pod \"maas-api-66b9899c65-krpz5\" (UID: \"ba2ff9d9-e91e-4d30-9d19-27a609e42447\") " pod="odh-ai-gateway-infra/maas-api-66b9899c65-krpz5"
Sep 06 11:03:10.209354 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.209319    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-openshift-service-ca\") pod \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\" (UID: \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\") "
Sep 06 11:03:10.209559 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.209368    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqftk\" (UniqueName: \"kubernetes.io/projected/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-kube-api-access-qqftk\") pod \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\" (UID: \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\") "
Sep 06 11:03:10.209559 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.209399    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-metrics-certs\") pod \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\" (UID: \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\") "
Sep 06 11:03:10.209559 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.209422    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-maas-api-tls\") pod \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\" (UID: \"9ed5d2da-6e3d-4547-8ba4-e7397ae9276c\") "
Sep 06 11:03:10.209819 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.209792    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9ed5d2da-6e3d-4547-8ba4-e7397ae9276c" (UID: "9ed5d2da-6e3d-4547-8ba4-e7397ae9276c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 11:03:10.211666 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.211642    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-kube-api-access-qqftk" (OuterVolumeSpecName: "kube-api-access-qqftk") pod "9ed5d2da-6e3d-4547-8ba4-e7397ae9276c" (UID: "9ed5d2da-6e3d-4547-8ba4-e7397ae9276c"). InnerVolumeSpecName "kube-api-access-qqftk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 11:03:10.211794 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.211663    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9ed5d2da-6e3d-4547-8ba4-e7397ae9276c" (UID: "9ed5d2da-6e3d-4547-8ba4-e7397ae9276c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 11:03:10.211794 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.211678    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9ed5d2da-6e3d-4547-8ba4-e7397ae9276c" (UID: "9ed5d2da-6e3d-4547-8ba4-e7397ae9276c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 11:03:10.247078 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.247052    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66b9899c65-krpz5"
Sep 06 11:03:10.311144 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.311069    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-openshift-service-ca\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 11:03:10.311144 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.311097    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqftk\" (UniqueName: \"kubernetes.io/projected/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-kube-api-access-qqftk\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 11:03:10.311144 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.311107    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-metrics-certs\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 11:03:10.311144 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.311116    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c-maas-api-tls\") on node \"ip-10-0-133-153.ec2.internal\" DevicePath \"\""
Sep 06 11:03:10.518816 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.518793    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-66b9899c65-krpz5"]
Sep 06 11:03:10.520625 ip-10-0-133-153 kubenswrapper[2557]: W0906 11:03:10.520596    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba2ff9d9_e91e_4d30_9d19_27a609e42447.slice/crio-96fec0e30df27b75ba144268c4cc0d2b1ddc9b116d964716f7784fa78c2146bf WatchSource:0}: Error finding container 96fec0e30df27b75ba144268c4cc0d2b1ddc9b116d964716f7784fa78c2146bf: Status 404 returned error can't find the container with id 96fec0e30df27b75ba144268c4cc0d2b1ddc9b116d964716f7784fa78c2146bf
Sep 06 11:03:10.859570 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:10.859524    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 11:03:11.104332 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:11.104298    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66b9899c65-krpz5" event={"ID":"ba2ff9d9-e91e-4d30-9d19-27a609e42447","Type":"ContainerStarted","Data":"6c69fef2cada5a6c8ae605d3703a1a37838f9d6b54a5e9ff28941eb48d558e31"}
Sep 06 11:03:11.104475 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:11.104347    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66b9899c65-krpz5" event={"ID":"ba2ff9d9-e91e-4d30-9d19-27a609e42447","Type":"ContainerStarted","Data":"96fec0e30df27b75ba144268c4cc0d2b1ddc9b116d964716f7784fa78c2146bf"}
Sep 06 11:03:11.104475 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:11.104367    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-66b9899c65-krpz5"
Sep 06 11:03:11.104475 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:11.104432    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"
Sep 06 11:03:11.123577 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:11.123535    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-66b9899c65-krpz5" podStartSLOduration=1.734052003 podStartE2EDuration="2.123521737s" totalImagesPullingTime="389.469734ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 11:03:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 11:03:11.122088568 +0000 UTC m=+2003.571869335" watchObservedRunningTime="2026-09-06 11:03:11.123521737 +0000 UTC m=+2003.573302506"
Sep 06 11:03:11.136739 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:11.136715    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"]
Sep 06 11:03:11.141681 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:11.141662    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-bd6c55749-hxsqh"]
Sep 06 11:03:12.038085 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:12.038050    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ed5d2da-6e3d-4547-8ba4-e7397ae9276c" path="/var/lib/kubelet/pods/9ed5d2da-6e3d-4547-8ba4-e7397ae9276c/volumes"
Sep 06 11:03:17.117014 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:17.116983    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-66b9899c65-krpz5"
Sep 06 11:03:19.808140 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:19.808101    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 11:03:27.808441 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:27.808406    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 11:03:34.914798 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:34.914761    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-k6wsv"]
Sep 06 11:03:42.673126 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:42.673099    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wghzk_54d0d5eb-abf2-46a4-8abf-b39d69b35a21/manager/0.log"
Sep 06 11:03:43.389059 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:43.389029    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-645f664568-rf25s_015b1b2b-33d7-4c34-9416-bd9a64fdc040/copy-manifests/0.log"
Sep 06 11:03:43.394890 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:43.394865    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-645f664568-rf25s_015b1b2b-33d7-4c34-9416-bd9a64fdc040/manager/0.log"
Sep 06 11:03:43.951015 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:43.950990    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-66b9899c65-krpz5_ba2ff9d9-e91e-4d30-9d19-27a609e42447/maas-api/0.log"
Sep 06 11:03:45.135978 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:45.135944    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-zg5p8_03d6634e-1105-4bf0-abf2-70c79c9e15d1/manager/0.log"
Sep 06 11:03:45.249505 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:45.249466    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-kj7ht_f1e5fbce-78b9-44cc-a76a-dc64c25c0780/manager/0.log"
Sep 06 11:03:45.481398 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:45.481323    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-xsj2q_2f028ca6-f40a-4374-84ae-59315b311745/manager/0.log"
Sep 06 11:03:45.614064 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:45.614034    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-k6wsv_4a59eacd-9266-4b63-a38d-8363c1378636/limitador/0.log"
Sep 06 11:03:45.732905 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:45.732843    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-jwtd2_befe56cc-b780-45e3-ae3f-39dc7988bc93/manager/0.log"
Sep 06 11:03:46.308218 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:46.308188    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5586db6db5-hp98n_9e1ab4e3-c725-4ae9-8e94-0f0739e10460/kube-auth-proxy/0.log"
Sep 06 11:03:46.528614 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:46.528587    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cqglj_8e05e829-48be-442d-8e9b-715b4604dc63/istio-proxy/1.log"
Sep 06 11:03:46.878652 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:46.878626    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78f4854998-kj5tp_0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c/router/0.log"
Sep 06 11:03:47.446862 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:47.446805    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-f5plp_df2c0194-ef33-407c-aea4-7ac2671506bc/main/0.log"
Sep 06 11:03:47.768292 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:47.768214    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-dx5fd_9b76b8b8-f168-4f89-8da8-79b479134957/main/0.log"
Sep 06 11:03:47.869753 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:47.869731    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-gwwms_7b9e9d5f-593e-4538-89aa-5110f37e8eb1/main/0.log"
Sep 06 11:03:48.593322 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:48.593286    2557 scope.go:117] "RemoveContainer" containerID="e38ca8ef4eef1462da250846397cd8de68d08621f5e7e550ca81c8dfaedea2e9"
Sep 06 11:03:48.601698 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:48.601680    2557 scope.go:117] "RemoveContainer" containerID="08ef3e195493b0f304aa4ed3c38ba19d3380e554abc395b10ff5d8c2136ec546"
Sep 06 11:03:55.977180 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:55.977137    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hbgj5_503a2587-73cc-43a0-b058-242b6b7ec81b/global-pull-secret-syncer/0.log"
Sep 06 11:03:56.128010 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:56.127981    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-kpqgt_1fdbdca8-792e-4abe-8058-3cc6df71bedd/konnectivity-agent/0.log"
Sep 06 11:03:56.199720 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:03:56.199694    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-153.ec2.internal_b6ddc99b2f4bb71ea7246abe1dc030e6/haproxy/0.log"
Sep 06 11:04:01.504347 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:01.504315    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-zg5p8_03d6634e-1105-4bf0-abf2-70c79c9e15d1/manager/0.log"
Sep 06 11:04:01.528311 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:01.528290    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-kj7ht_f1e5fbce-78b9-44cc-a76a-dc64c25c0780/manager/0.log"
Sep 06 11:04:01.608155 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:01.608131    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-xsj2q_2f028ca6-f40a-4374-84ae-59315b311745/manager/0.log"
Sep 06 11:04:01.649181 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:01.649158    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-k6wsv_4a59eacd-9266-4b63-a38d-8363c1378636/limitador/0.log"
Sep 06 11:04:01.695027 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:01.695006    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-jwtd2_befe56cc-b780-45e3-ae3f-39dc7988bc93/manager/0.log"
Sep 06 11:04:03.305107 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:03.305076    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-zmk6j_4e55abd8-3289-4fff-8d8f-34129da7fb69/cluster-monitoring-operator/0.log"
Sep 06 11:04:03.520665 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:03.520639    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-286w4_128c863a-c6d4-483e-ac5a-dc38cb426de3/node-exporter/0.log"
Sep 06 11:04:03.547623 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:03.547596    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-286w4_128c863a-c6d4-483e-ac5a-dc38cb426de3/kube-rbac-proxy/0.log"
Sep 06 11:04:03.574248 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:03.574176    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-286w4_128c863a-c6d4-483e-ac5a-dc38cb426de3/init-textfile/0.log"
Sep 06 11:04:05.677565 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:05.677539    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-qh488_2b5a6c6a-2f52-4d65-8000-c39f44942495/networking-console-plugin/0.log"
Sep 06 11:04:06.268711 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.268680    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/2.log"
Sep 06 11:04:06.272461 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.272441    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9lt24_938e293f-7414-4bdd-946f-0b463ddc8ecb/console-operator/3.log"
Sep 06 11:04:06.560614 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.560542    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"]
Sep 06 11:04:06.560904 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.560893    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ed5d2da-6e3d-4547-8ba4-e7397ae9276c" containerName="maas-api"
Sep 06 11:04:06.560948 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.560906    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ed5d2da-6e3d-4547-8ba4-e7397ae9276c" containerName="maas-api"
Sep 06 11:04:06.560987 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.560977    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ed5d2da-6e3d-4547-8ba4-e7397ae9276c" containerName="maas-api"
Sep 06 11:04:06.564045 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.564030    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"
Sep 06 11:04:06.566336 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.566315    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pw9j6\"/\"default-dockercfg-xq2z4\""
Sep 06 11:04:06.566472 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.566317    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pw9j6\"/\"openshift-service-ca.crt\""
Sep 06 11:04:06.567220 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.567201    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pw9j6\"/\"kube-root-ca.crt\""
Sep 06 11:04:06.571318 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.571193    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"]
Sep 06 11:04:06.687640 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.687613    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t95bt\" (UniqueName: \"kubernetes.io/projected/f2e18143-efd7-4d65-9e66-976846d7fc9f-kube-api-access-t95bt\") pod \"perf-node-gather-daemonset-92r5l\" (UID: \"f2e18143-efd7-4d65-9e66-976846d7fc9f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"
Sep 06 11:04:06.687945 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.687646    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f2e18143-efd7-4d65-9e66-976846d7fc9f-proc\") pod \"perf-node-gather-daemonset-92r5l\" (UID: \"f2e18143-efd7-4d65-9e66-976846d7fc9f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"
Sep 06 11:04:06.687945 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.687667    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f2e18143-efd7-4d65-9e66-976846d7fc9f-podres\") pod \"perf-node-gather-daemonset-92r5l\" (UID: \"f2e18143-efd7-4d65-9e66-976846d7fc9f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"
Sep 06 11:04:06.687945 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.687706    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2e18143-efd7-4d65-9e66-976846d7fc9f-lib-modules\") pod \"perf-node-gather-daemonset-92r5l\" (UID: \"f2e18143-efd7-4d65-9e66-976846d7fc9f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"
Sep 06 11:04:06.687945 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.687783    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2e18143-efd7-4d65-9e66-976846d7fc9f-sys\") pod \"perf-node-gather-daemonset-92r5l\" (UID: \"f2e18143-efd7-4d65-9e66-976846d7fc9f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"
Sep 06 11:04:06.788831 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.788803    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2e18143-efd7-4d65-9e66-976846d7fc9f-sys\") pod \"perf-node-gather-daemonset-92r5l\" (UID: \"f2e18143-efd7-4d65-9e66-976846d7fc9f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"
Sep 06 11:04:06.788956 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.788846    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t95bt\" (UniqueName: \"kubernetes.io/projected/f2e18143-efd7-4d65-9e66-976846d7fc9f-kube-api-access-t95bt\") pod \"perf-node-gather-daemonset-92r5l\" (UID: \"f2e18143-efd7-4d65-9e66-976846d7fc9f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"
Sep 06 11:04:06.788956 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.788875    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f2e18143-efd7-4d65-9e66-976846d7fc9f-proc\") pod \"perf-node-gather-daemonset-92r5l\" (UID: \"f2e18143-efd7-4d65-9e66-976846d7fc9f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"
Sep 06 11:04:06.788956 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.788899    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f2e18143-efd7-4d65-9e66-976846d7fc9f-podres\") pod \"perf-node-gather-daemonset-92r5l\" (UID: \"f2e18143-efd7-4d65-9e66-976846d7fc9f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"
Sep 06 11:04:06.788956 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.788925    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2e18143-efd7-4d65-9e66-976846d7fc9f-sys\") pod \"perf-node-gather-daemonset-92r5l\" (UID: \"f2e18143-efd7-4d65-9e66-976846d7fc9f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"
Sep 06 11:04:06.788956 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.788933    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2e18143-efd7-4d65-9e66-976846d7fc9f-lib-modules\") pod \"perf-node-gather-daemonset-92r5l\" (UID: \"f2e18143-efd7-4d65-9e66-976846d7fc9f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"
Sep 06 11:04:06.789205 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.789001    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f2e18143-efd7-4d65-9e66-976846d7fc9f-proc\") pod \"perf-node-gather-daemonset-92r5l\" (UID: \"f2e18143-efd7-4d65-9e66-976846d7fc9f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"
Sep 06 11:04:06.789205 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.789032    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2e18143-efd7-4d65-9e66-976846d7fc9f-lib-modules\") pod \"perf-node-gather-daemonset-92r5l\" (UID: \"f2e18143-efd7-4d65-9e66-976846d7fc9f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"
Sep 06 11:04:06.789205 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.789041    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f2e18143-efd7-4d65-9e66-976846d7fc9f-podres\") pod \"perf-node-gather-daemonset-92r5l\" (UID: \"f2e18143-efd7-4d65-9e66-976846d7fc9f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"
Sep 06 11:04:06.795817 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.795797    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t95bt\" (UniqueName: \"kubernetes.io/projected/f2e18143-efd7-4d65-9e66-976846d7fc9f-kube-api-access-t95bt\") pod \"perf-node-gather-daemonset-92r5l\" (UID: \"f2e18143-efd7-4d65-9e66-976846d7fc9f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"
Sep 06 11:04:06.875379 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:06.875357    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"
Sep 06 11:04:07.142120 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:07.142096    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"]
Sep 06 11:04:07.144241 ip-10-0-133-153 kubenswrapper[2557]: W0906 11:04:07.144216    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf2e18143_efd7_4d65_9e66_976846d7fc9f.slice/crio-a9eee11a60268245e8f2a3a5849aaf34da79adf289fee4b37452e2d82d36aeed WatchSource:0}: Error finding container a9eee11a60268245e8f2a3a5849aaf34da79adf289fee4b37452e2d82d36aeed: Status 404 returned error can't find the container with id a9eee11a60268245e8f2a3a5849aaf34da79adf289fee4b37452e2d82d36aeed
Sep 06 11:04:07.387605 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:07.387565    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l" event={"ID":"f2e18143-efd7-4d65-9e66-976846d7fc9f","Type":"ContainerStarted","Data":"1ea4f6e78fb97de410ec2216aa2e9a455f002b6f4ab1520143db71ac05473fa4"}
Sep 06 11:04:07.387781 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:07.387675    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l" event={"ID":"f2e18143-efd7-4d65-9e66-976846d7fc9f","Type":"ContainerStarted","Data":"a9eee11a60268245e8f2a3a5849aaf34da79adf289fee4b37452e2d82d36aeed"}
Sep 06 11:04:07.387859 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:07.387808    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"
Sep 06 11:04:07.405036 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:07.404961    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l" podStartSLOduration=1.404947094 podStartE2EDuration="1.404947094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 11:04:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 11:04:07.401964024 +0000 UTC m=+2059.851744794" watchObservedRunningTime="2026-09-06 11:04:07.404947094 +0000 UTC m=+2059.854727863"
Sep 06 11:04:07.550711 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:07.550686    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-tr4gb_9dc8c2c3-d63b-4ae4-8458-e37779164ff2/volume-data-source-validator/0.log"
Sep 06 11:04:08.732322 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:08.732296    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dltkp_ec03af21-0135-4ba1-b4be-06adf19748ed/dns/0.log"
Sep 06 11:04:08.750949 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:08.750917    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dltkp_ec03af21-0135-4ba1-b4be-06adf19748ed/kube-rbac-proxy/0.log"
Sep 06 11:04:08.861437 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:08.861408    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t57dx_8a42dde3-cdd1-49a0-86f1-c67b65ca9cee/dns-node-resolver/0.log"
Sep 06 11:04:09.477154 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:09.477127    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-st44k_d4a9d66d-ae69-42c7-a978-1cdfdcc06fd3/node-ca/0.log"
Sep 06 11:04:10.936992 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:10.936916    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5586db6db5-hp98n_9e1ab4e3-c725-4ae9-8e94-0f0739e10460/kube-auth-proxy/0.log"
Sep 06 11:04:10.988565 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:10.988539    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cqglj_8e05e829-48be-442d-8e9b-715b4604dc63/istio-proxy/0.log"
Sep 06 11:04:10.997159 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:10.997133    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cqglj_8e05e829-48be-442d-8e9b-715b4604dc63/istio-proxy/1.log"
Sep 06 11:04:11.107077 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:11.107052    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78f4854998-kj5tp_0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c/router/0.log"
Sep 06 11:04:11.952320 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:11.952292    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-74t29_0f62f367-e8fd-4f68-95ba-50376d52e32a/serve-healthcheck-canary/0.log"
Sep 06 11:04:12.466766 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:12.466743    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-vnd8g_679d9837-653d-4ac4-b786-dd55ac8a5e06/insights-operator/0.log"
Sep 06 11:04:12.467307 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:12.467288    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-vnd8g_679d9837-653d-4ac4-b786-dd55ac8a5e06/insights-operator/1.log"
Sep 06 11:04:12.615699 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:12.615675    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pfxj6_dace2beb-5038-4b5b-936c-120b52c07ed4/kube-rbac-proxy/0.log"
Sep 06 11:04:12.634751 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:12.634729    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pfxj6_dace2beb-5038-4b5b-936c-120b52c07ed4/exporter/0.log"
Sep 06 11:04:12.653891 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:12.653871    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pfxj6_dace2beb-5038-4b5b-936c-120b52c07ed4/extractor/0.log"
Sep 06 11:04:13.405227 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:13.405198    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-92r5l"
Sep 06 11:04:14.798651 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:14.798614    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wghzk_54d0d5eb-abf2-46a4-8abf-b39d69b35a21/manager/0.log"
Sep 06 11:04:15.404762 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:15.404716    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-645f664568-rf25s_015b1b2b-33d7-4c34-9416-bd9a64fdc040/manager/0.log"
Sep 06 11:04:15.430652 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:15.430626    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-645f664568-rf25s_015b1b2b-33d7-4c34-9416-bd9a64fdc040/copy-manifests/0.log"
Sep 06 11:04:17.757366 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:17.757338    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5586db6db5-hp98n_9e1ab4e3-c725-4ae9-8e94-0f0739e10460/kube-auth-proxy/0.log"
Sep 06 11:04:17.790450 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:17.790381    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cqglj_8e05e829-48be-442d-8e9b-715b4604dc63/istio-proxy/0.log"
Sep 06 11:04:17.800468 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:17.800445    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cqglj_8e05e829-48be-442d-8e9b-715b4604dc63/istio-proxy/1.log"
Sep 06 11:04:17.898625 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:17.898597    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78f4854998-kj5tp_0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c/router/0.log"
Sep 06 11:04:19.928800 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:19.928772    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5586db6db5-hp98n_9e1ab4e3-c725-4ae9-8e94-0f0739e10460/kube-auth-proxy/0.log"
Sep 06 11:04:19.962712 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:19.962689    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cqglj_8e05e829-48be-442d-8e9b-715b4604dc63/istio-proxy/0.log"
Sep 06 11:04:19.976510 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:19.976468    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cqglj_8e05e829-48be-442d-8e9b-715b4604dc63/istio-proxy/1.log"
Sep 06 11:04:20.078024 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:20.077979    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78f4854998-kj5tp_0e89829b-4d1d-4a6a-b4d1-ef2601dbcd0c/router/0.log"
Sep 06 11:04:21.894566 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:21.894539    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-k9kzz_840d1c17-0109-4541-93cb-7409823e8f70/migrator/0.log"
Sep 06 11:04:21.912630 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:21.912609    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-k9kzz_840d1c17-0109-4541-93cb-7409823e8f70/graceful-termination/0.log"
Sep 06 11:04:22.261315 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:22.261246    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-7jr4k_99fa6d2e-adc6-4bd8-b6b0-e73045c36405/kube-storage-version-migrator-operator/1.log"
Sep 06 11:04:22.262187 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:22.262166    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-7jr4k_99fa6d2e-adc6-4bd8-b6b0-e73045c36405/kube-storage-version-migrator-operator/0.log"
Sep 06 11:04:23.577311 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:23.577286    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l57sp_b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3/kube-multus-additional-cni-plugins/0.log"
Sep 06 11:04:23.596630 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:23.596604    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l57sp_b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3/egress-router-binary-copy/0.log"
Sep 06 11:04:23.617610 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:23.617587    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l57sp_b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3/cni-plugins/0.log"
Sep 06 11:04:23.636530 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:23.636481    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l57sp_b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3/bond-cni-plugin/0.log"
Sep 06 11:04:23.656470 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:23.656451    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l57sp_b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3/routeoverride-cni/0.log"
Sep 06 11:04:23.677245 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:23.677229    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l57sp_b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3/whereabouts-cni-bincopy/0.log"
Sep 06 11:04:23.698602 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:23.698585    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l57sp_b3d2bc45-9a8b-4efc-83ec-c5f57d5df6f3/whereabouts-cni/0.log"
Sep 06 11:04:23.740921 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:23.740898    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-z72zd_f27bdb46-dbcb-4639-ad28-3ed384e21ba6/kube-multus/0.log"
Sep 06 11:04:23.803449 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:23.803427    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pgxvx_5ef2cd14-0fae-417c-8d8a-b407e0c03903/network-metrics-daemon/0.log"
Sep 06 11:04:23.820433 ip-10-0-133-153 kubenswrapper[2557]: I0906 11:04:23.820415    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pgxvx_5ef2cd14-0fae-417c-8d8a-b407e0c03903/kube-rbac-proxy/0.log"