Jul 28 14:16:45.556492 ip-10-0-130-66 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 14:16:45.556501 ip-10-0-130-66 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 14:16:45.556509 ip-10-0-130-66 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 14:16:45.556742 ip-10-0-130-66 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 14:16:55.585201 ip-10-0-130-66 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 14:16:55.585222 ip-10-0-130-66 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 0b08970b7c37475eae0d624f6026bce3 --
Jul 28 14:18:50.090405 ip-10-0-130-66 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 14:18:50.495262 ip-10-0-130-66 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 14:18:50.495262 ip-10-0-130-66 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 14:18:50.495262 ip-10-0-130-66 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 14:18:50.495262 ip-10-0-130-66 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 14:18:50.495262 ip-10-0-130-66 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 14:18:50.497409 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.497313    2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 14:18:50.503548 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503526    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 14:18:50.503548 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503547    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503552    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503556    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503559    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503562    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503567    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503571    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503575    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503578    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503581    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503584    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503587    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503590    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503593    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503596    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503600    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503603    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503605    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503608    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503611    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 14:18:50.503625 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503614    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 14:18:50.504197 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503617    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 14:18:50.504197 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503620    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 14:18:50.504197 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503623    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 14:18:50.504197 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503626    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 14:18:50.504197 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503629    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 14:18:50.504197 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503632    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 14:18:50.504197 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503635    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 14:18:50.504197 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503638    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 14:18:50.504197 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503641    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 14:18:50.504197 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503644    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 14:18:50.504197 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503647    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 14:18:50.504197 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503650    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 14:18:50.504197 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503653    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 14:18:50.504197 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503655    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 14:18:50.504197 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503658    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 14:18:50.504197 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503661    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 14:18:50.504197 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503666    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 14:18:50.504197 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503670    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 14:18:50.504197 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503672    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 14:18:50.504709 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503676    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 14:18:50.504709 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503678    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 14:18:50.504709 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503681    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 14:18:50.504709 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503684    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 14:18:50.504709 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503687    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 14:18:50.504709 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503689    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 14:18:50.504709 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503692    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 14:18:50.504709 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503695    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 14:18:50.504709 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503702    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 14:18:50.504709 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503705    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 14:18:50.504709 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503708    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 14:18:50.504709 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503711    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 14:18:50.504709 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503713    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 14:18:50.504709 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503716    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 14:18:50.504709 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503720    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 14:18:50.504709 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503722    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 14:18:50.504709 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503725    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 14:18:50.504709 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503728    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 14:18:50.504709 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503731    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503734    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503737    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503740    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503742    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503745    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503748    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503751    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503753    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503757    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503759    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503762    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503765    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503768    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503771    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503774    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503777    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503779    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503782    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503785    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 14:18:50.505498 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503788    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503791    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503793    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503797    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503801    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503804    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503807    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503810    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503815    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503819    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503822    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503825    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503828    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503831    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503834    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503837    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503839    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503842    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503845    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.503848    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 14:18:50.506339 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505403    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 14:18:50.507254 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505429    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 14:18:50.507254 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505436    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 14:18:50.507254 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505442    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 14:18:50.507254 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505448    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 14:18:50.507254 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505452    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 14:18:50.507254 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505457    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 14:18:50.507254 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505462    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 14:18:50.507254 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505466    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 14:18:50.507254 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505471    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 14:18:50.507254 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505482    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 14:18:50.507254 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505486    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 14:18:50.507254 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505490    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 14:18:50.507254 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505495    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 14:18:50.507254 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505500    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 14:18:50.507254 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505505    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 14:18:50.507254 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505510    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 14:18:50.507254 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505515    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 14:18:50.507254 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505519    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 14:18:50.507254 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505524    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 14:18:50.508150 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505531    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 14:18:50.508150 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505539    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 14:18:50.508150 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505551    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 14:18:50.508150 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505556    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 14:18:50.508150 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505563    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 14:18:50.508150 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505570    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 14:18:50.508150 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505575    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 14:18:50.508150 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505580    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 14:18:50.508150 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505585    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 14:18:50.508150 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505589    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 14:18:50.508150 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505593    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 14:18:50.508150 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505598    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 14:18:50.508150 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505602    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 14:18:50.508150 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505609    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 14:18:50.508150 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505618    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 14:18:50.508150 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505623    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 14:18:50.508150 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505630    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 14:18:50.508150 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505635    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 14:18:50.508150 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505639    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505643    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505647    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505652    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505656    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505661    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505666    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505671    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505680    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505684    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505689    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505693    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505698    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505702    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505707    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505711    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505716    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505720    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505803    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505812    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 14:18:50.509057 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505819    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505826    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505832    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505838    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505844    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505850    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505856    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505862    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505883    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505896    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505901    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.505912    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506074    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506087    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506107    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506112    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506116    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506121    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506126    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506130    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 14:18:50.509766 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506136    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 14:18:50.510558 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506140    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 14:18:50.510558 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506145    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 14:18:50.510558 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506149    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 14:18:50.510558 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506158    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 14:18:50.510558 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506165    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 14:18:50.510558 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506170    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 14:18:50.510558 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506175    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 14:18:50.510558 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506180    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 14:18:50.510558 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506185    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 14:18:50.510558 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506191    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 14:18:50.510558 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506195    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 14:18:50.510558 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506200    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 14:18:50.510558 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506205    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 14:18:50.510558 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506210    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 14:18:50.510558 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506214    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 14:18:50.510558 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506219    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 14:18:50.510558 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506223    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 14:18:50.510558 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506228    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 14:18:50.510558 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506233    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506237    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506960    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506970    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506975    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506980    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506985    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506989    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506995    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.506999    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507004    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507009    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507014    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507019    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507024    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507029    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507033    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507038    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507042    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507047    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 14:18:50.511103 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507052    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 14:18:50.511729 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507058    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 14:18:50.511729 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507062    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 14:18:50.511729 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507071    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 14:18:50.511729 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507075    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 14:18:50.511729 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507081    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 14:18:50.511729 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507087    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 14:18:50.511729 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507093    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 14:18:50.511729 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507097    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 14:18:50.511729 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507102    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 14:18:50.511729 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507107    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 14:18:50.511729 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507111    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 14:18:50.511729 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507116    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 14:18:50.511729 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507121    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 14:18:50.511729 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507126    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 14:18:50.511729 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507130    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 14:18:50.511729 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507135    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 14:18:50.511729 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507140    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 14:18:50.511729 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507144    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 14:18:50.511729 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507153    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 14:18:50.512313 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507160    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 14:18:50.512313 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507165    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 14:18:50.512313 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507184    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 14:18:50.512313 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507190    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 14:18:50.512313 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507195    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 14:18:50.512313 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507199    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 14:18:50.512313 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507204    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 14:18:50.512313 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507208    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 14:18:50.512313 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507213    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 14:18:50.512313 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507217    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 14:18:50.512313 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507222    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 14:18:50.512313 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507228    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 14:18:50.512313 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507233    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 14:18:50.512313 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507238    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 14:18:50.512313 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507243    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 14:18:50.512313 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507250    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 14:18:50.512313 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507254    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 14:18:50.512313 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507259    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 14:18:50.512313 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507263    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507267    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507272    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507277    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507281    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507286    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507291    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507296    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507301    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507305    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507310    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507315    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507319    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507324    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507329    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507336    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507340    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507345    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507352    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507358    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 14:18:50.512822 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507363    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507368    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507373    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507379    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507385    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507391    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507397    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507402    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507406    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507414    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507418    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507423    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507427    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507432    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507437    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507441    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507446    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507451    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507455    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507459    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 14:18:50.513437 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507464    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507468    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.507472    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507578    2584 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507592    2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507603    2584 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507612    2584 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507620    2584 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507626    2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507633    2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507640    2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507646    2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507652    2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507658    2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507664    2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507670    2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507675    2584 flags.go:64] FLAG: --cgroup-root=""
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507680    2584 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507686    2584 flags.go:64] FLAG: --client-ca-file=""
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507692    2584 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507697    2584 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507707    2584 flags.go:64] FLAG: --cluster-domain=""
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507712    2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 14:18:50.513960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507718    2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507723    2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507729    2584 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507736    2584 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507741    2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507747    2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507752    2584 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507758    2584 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507763    2584 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507768    2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507773    2584 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507778    2584 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507786    2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507791    2584 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507797    2584 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507802    2584 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507807    2584 flags.go:64] FLAG: --enable-server="true"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507812    2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507820    2584 flags.go:64] FLAG: --event-burst="100"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507825    2584 flags.go:64] FLAG: --event-qps="50"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507830    2584 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507836    2584 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507841    2584 flags.go:64] FLAG: --eviction-hard=""
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507849    2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 14:18:50.514587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507854    2584 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507859    2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507865    2584 flags.go:64] FLAG: --eviction-soft=""
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507870    2584 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507875    2584 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507881    2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507888    2584 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507894    2584 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507899    2584 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507904    2584 flags.go:64] FLAG: --feature-gates=""
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507911    2584 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507916    2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507921    2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507927    2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507932    2584 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507938    2584 flags.go:64] FLAG: --help="false"
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507943    2584 flags.go:64] FLAG: --hostname-override="ip-10-0-130-66.ec2.internal"
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507949    2584 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507954    2584 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507959    2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507965    2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507971    2584 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507976    2584 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 14:18:50.515316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507982    2584 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507987    2584 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507992    2584 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.507997    2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508003    2584 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508009    2584 flags.go:64] FLAG: --kube-reserved=""
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508014    2584 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508019    2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508027    2584 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508032    2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508038    2584 flags.go:64] FLAG: --lock-file=""
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508043    2584 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508048    2584 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508053    2584 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508063    2584 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508071    2584 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508077    2584 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508082    2584 flags.go:64] FLAG: --logging-format="text"
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508087    2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508093    2584 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508098    2584 flags.go:64] FLAG: --manifest-url=""
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508103    2584 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508111    2584 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508116    2584 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508124    2584 flags.go:64] FLAG: --max-pods="110"
Jul 28 14:18:50.515965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508129    2584 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508134    2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508139    2584 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508145    2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508150    2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508155    2584 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508160    2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508190    2584 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508195    2584 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508200    2584 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508206    2584 flags.go:64] FLAG: --pod-cidr=""
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508211    2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508220    2584 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508225    2584 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508230    2584 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508236    2584 flags.go:64] FLAG: --port="10250"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508243    2584 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508248    2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08eafacc8337265ac"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508254    2584 flags.go:64] FLAG: --qos-reserved=""
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508259    2584 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508265    2584 flags.go:64] FLAG: --register-node="true"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508270    2584 flags.go:64] FLAG: --register-with-taints=""
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508279    2584 flags.go:64] FLAG: --registry-burst="10"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508285    2584 flags.go:64] FLAG: --registry-qps="5"
Jul 28 14:18:50.516652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508291    2584 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508296    2584 flags.go:64] FLAG: --reserved-memory=""
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508303    2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508308    2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508313    2584 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508319    2584 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508323    2584 flags.go:64] FLAG: --runonce="false"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508329    2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508334    2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508339    2584 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508344    2584 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508349    2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508355    2584 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508360    2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508365    2584 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508370    2584 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508375    2584 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508380    2584 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508386    2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508392    2584 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508397    2584 flags.go:64] FLAG: --system-cgroups=""
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508402    2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508411    2584 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508416    2584 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508424    2584 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 14:18:50.517367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508433    2584 flags.go:64] FLAG: --tls-min-version=""
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508438    2584 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508443    2584 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508448    2584 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508454    2584 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508460    2584 flags.go:64] FLAG: --v="2"
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508468    2584 flags.go:64] FLAG: --version="false"
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508481    2584 flags.go:64] FLAG: --vmodule=""
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508488    2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.508494    2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508701    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508708    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508713    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508721    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508727    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508733    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508737    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508743    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508748    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508753    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508758    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 14:18:50.518017 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508762    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508767    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508772    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508777    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508781    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508786    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508791    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508795    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508800    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508805    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508812    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508816    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508821    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508826    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508831    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508836    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508842    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508847    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508852    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508859    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 14:18:50.518644 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508863    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508868    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508872    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508877    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508882    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508886    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508891    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508896    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508900    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508905    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508909    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508914    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508918    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508922    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508927    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508932    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508936    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508941    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508946    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508950    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 14:18:50.519219 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508955    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 14:18:50.519732 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508959    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 14:18:50.519732 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508966    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 14:18:50.519732 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508971    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 14:18:50.519732 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508976    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 14:18:50.519732 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508981    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 14:18:50.519732 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508985    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 14:18:50.519732 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508990    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 14:18:50.519732 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.508996    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 14:18:50.519732 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509000    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 14:18:50.519732 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509005    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 14:18:50.519732 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509009    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 14:18:50.519732 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509018    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 14:18:50.519732 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509024    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 14:18:50.519732 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509029    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 14:18:50.519732 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509035    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 14:18:50.519732 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509040    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 14:18:50.519732 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509044    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 14:18:50.519732 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509050    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 14:18:50.519732 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509055    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509059    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509064    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509068    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509073    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509077    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509081    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509086    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509091    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509095    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509100    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509105    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509110    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509117    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509123    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509129    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509134    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509139    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509145    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509150    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 14:18:50.520271 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509155    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 14:18:50.520799 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509160    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 14:18:50.520799 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509182    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 14:18:50.520799 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509187    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 14:18:50.520799 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509191    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 14:18:50.520799 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509195    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 14:18:50.520799 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509201    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 14:18:50.520799 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509206    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 14:18:50.520799 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.509210    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 14:18:50.520799 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.509220    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 14:18:50.520799 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.516849    2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 14:18:50.520799 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.516866    2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 14:18:50.520799 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.517919    2584 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 14:18:50.520799 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.517933    2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 14:18:50.520799 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.517998    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 14:18:50.521191 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518004    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 14:18:50.521191 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518007    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 14:18:50.521191 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518010    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 14:18:50.521191 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518013    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 14:18:50.521191 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518016    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 14:18:50.521191 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518020    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 14:18:50.521191 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518023    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 14:18:50.521191 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518026    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 14:18:50.521191 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518029    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 14:18:50.521191 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518033    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 14:18:50.521191 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518035    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 14:18:50.521191 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518038    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 14:18:50.521191 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518041    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 14:18:50.521191 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518044    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 14:18:50.521191 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518046    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 14:18:50.521191 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518049    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 14:18:50.521191 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518052    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 14:18:50.521191 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518054    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 14:18:50.521191 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518059    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 14:18:50.521684 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518063    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 14:18:50.521684 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518066    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 14:18:50.521684 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518069    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 14:18:50.521684 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518073    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 14:18:50.521684 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518077    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 14:18:50.521684 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518080    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 14:18:50.521684 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518083    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 14:18:50.521684 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518086    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 14:18:50.521684 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518089    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 14:18:50.521684 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518093    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 14:18:50.521684 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518096    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 14:18:50.521684 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518099    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 14:18:50.521684 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518101    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 14:18:50.521684 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518121    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 14:18:50.521684 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518126    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 14:18:50.521684 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518130    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 14:18:50.521684 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518133    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 14:18:50.521684 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518136    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 14:18:50.521684 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518141    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518145    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518149    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518152    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518155    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518159    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518162    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518165    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518179    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518182    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518186    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518189    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518192    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518196    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518199    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518202    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518204    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518207    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518210    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518214    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 14:18:50.522249 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518217    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518220    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518223    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518227    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518230    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518233    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518236    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518238    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518241    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518245    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518249    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518252    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518255    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518258    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518261    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518263    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518266    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518269    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518272    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518274    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 14:18:50.522790 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518278    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518280    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518283    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518286    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518289    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518292    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518295    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518297    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518300    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518303    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518306    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518309    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518313    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518315    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518318    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518321    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518324    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518327    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518330    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518333    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 14:18:50.523340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518336    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 14:18:50.523855 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.518341    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 14:18:50.523855 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518469    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 14:18:50.523855 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518474    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 14:18:50.523855 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518478    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 14:18:50.523855 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518481    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 14:18:50.523855 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518484    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 14:18:50.523855 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518487    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 14:18:50.523855 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518490    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 14:18:50.523855 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518493    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 14:18:50.523855 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518496    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 14:18:50.523855 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518499    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 14:18:50.523855 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518502    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 14:18:50.523855 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518504    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 14:18:50.523855 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518507    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 14:18:50.523855 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518511    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518515    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518518    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518522    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518525    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518528    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518531    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518534    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518537    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518540    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518543    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518546    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518548    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518552    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518555    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518557    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518560    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518563    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518566    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518569    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 14:18:50.524267 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518572    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518575    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518579    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518582    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518585    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518588    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518591    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518593    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518596    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518599    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518602    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518604    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518607    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518610    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518612    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518616    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518619    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518622    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518624    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518627    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 14:18:50.524800 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518630    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518633    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518636    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518639    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518642    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518645    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518648    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518651    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518654    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518657    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518660    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518663    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518666    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518668    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518671    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518674    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518676    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518679    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518682    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518685    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 14:18:50.525340 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518687    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 14:18:50.525850 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518690    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 14:18:50.525850 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518693    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 14:18:50.525850 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518696    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 14:18:50.525850 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518699    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 14:18:50.525850 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518702    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 14:18:50.525850 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518704    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 14:18:50.525850 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518707    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 14:18:50.525850 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518710    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 14:18:50.525850 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518713    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 14:18:50.525850 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518716    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 14:18:50.525850 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518719    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 14:18:50.525850 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518721    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 14:18:50.525850 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518724    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 14:18:50.525850 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518728    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 14:18:50.525850 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518732    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 14:18:50.525850 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518735    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 14:18:50.525850 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518738    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 14:18:50.525850 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518742    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 14:18:50.525850 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518745    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 14:18:50.526423 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518748    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 14:18:50.526423 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518750    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 14:18:50.526423 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518753    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 14:18:50.526423 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518756    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 14:18:50.526423 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518758    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 14:18:50.526423 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:50.518761    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 14:18:50.526423 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.518767    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 14:18:50.526423 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.518902    2584 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 14:18:50.526423 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.521257    2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 14:18:50.526423 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.522781    2584 server.go:1019] "Starting client certificate rotation"
Jul 28 14:18:50.526423 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.522892    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 14:18:50.526423 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.522934    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 14:18:50.526423 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.525475    2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 14:18:50.526782 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.526768    2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 14:18:50.539498 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.539292    2584 log.go:25] "Validated CRI v1 runtime API"
Jul 28 14:18:50.542507 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.542488    2584 log.go:25] "Validated CRI v1 image API"
Jul 28 14:18:50.544152 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.544133    2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 14:18:50.546953 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.546930    2584 fs.go:135] Filesystem UUIDs: map[4b3ceffc-7b79-457a-9526-5502760c8043:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 eafe88e9-ae87-4de4-b0a4-6186ae1e1e96:/dev/nvme0n1p3]
Jul 28 14:18:50.547031 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.546952    2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 14:18:50.553159 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.553030    2584 manager.go:216] Machine: {Timestamp:2026-07-28 14:18:50.551352217 +0000 UTC m=+0.352894426 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096371 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2518a8cd9984be024ae7179cf291a0 SystemUUID:ec2518a8-cd99-84be-024a-e7179cf291a0 BootID:0b08970b-7c37-475e-ae0d-624f6026bce3 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2d:2b:b0:ec:cf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2d:2b:b0:ec:cf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:e1:24:79:56:1f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 14:18:50.553159 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.553149    2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 14:18:50.553294 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.553257    2584 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 14:18:50.556645 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.556615    2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 14:18:50.556809 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.556650    2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-66.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 14:18:50.557525 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.557511    2584 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 14:18:50.557561 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.557529    2584 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 14:18:50.557946 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.557935    2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 14:18:50.557990 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.557963    2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 14:18:50.558838 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.558827    2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 14:18:50.559266 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.559240    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 14:18:50.560922 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.560909    2584 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 14:18:50.561054 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.561043    2584 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 14:18:50.561092 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.561076    2584 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 14:18:50.561092 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.561086    2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 14:18:50.561148 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.561098    2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 14:18:50.561148 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.561109    2584 kubelet.go:398] "Adding apiserver pod source"
Jul 28 14:18:50.561148 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.561119    2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 14:18:50.561566 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.561554    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 14:18:50.561612 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.561577    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 14:18:50.564275 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.564255    2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 14:18:50.566316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.566300    2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 14:18:50.566383 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.566333    2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 14:18:50.567937 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.567919    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 14:18:50.567995 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.567948    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 14:18:50.567995 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.567959    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 14:18:50.567995 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.567975    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 14:18:50.567995 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.567993    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 14:18:50.568110 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.568004    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 14:18:50.568110 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.568016    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 14:18:50.568110 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.568022    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 14:18:50.568110 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.568031    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 14:18:50.568110 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.568038    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 14:18:50.568110 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.568047    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 14:18:50.568110 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.568053    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 14:18:50.568937 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.568923    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 14:18:50.568937 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.568935    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 14:18:50.571393 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:50.571369    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-66.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 14:18:50.571453 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:50.571370    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 14:18:50.571551 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.571534    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-66.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 14:18:50.572451 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.572437    2584 server.go:1268] "Started kubelet"
Jul 28 14:18:50.572561 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.572532    2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 14:18:50.573479 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.573423    2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 14:18:50.573570 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.573497    2584 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 14:18:50.573602 ip-10-0-130-66 systemd[1]: Started Kubernetes Kubelet.
Jul 28 14:18:50.578543 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.578480    2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 14:18:50.579792 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:50.579735    2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 14:18:50.580295 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.580276    2584 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 14:18:50.582987 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:50.581857    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-66.ec2.internal.18c679e5e3a09dfa  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-66.ec2.internal,UID:ip-10-0-130-66.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-66.ec2.internal,},FirstTimestamp:2026-07-28 14:18:50.572406266 +0000 UTC m=+0.373948474,LastTimestamp:2026-07-28 14:18:50.572406266 +0000 UTC m=+0.373948474,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-66.ec2.internal,}"
Jul 28 14:18:50.585686 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.585666    2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 14:18:50.585775 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.585685    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 14:18:50.585839 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:50.585797    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 28 14:18:50.586434 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.586412    2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 14:18:50.586534 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.586523    2584 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 14:18:50.586597 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.586538    2584 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 14:18:50.586715 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.586698    2584 factory.go:55] Registering systemd factory
Jul 28 14:18:50.586781 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.586721    2584 factory.go:223] Registration of the systemd container factory successfully
Jul 28 14:18:50.586781 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.586709    2584 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 14:18:50.586781 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.586762    2584 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 14:18:50.586973 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.586941    2584 factory.go:153] Registering CRI-O factory
Jul 28 14:18:50.586973 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.586961    2584 factory.go:223] Registration of the crio container factory successfully
Jul 28 14:18:50.587145 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.587021    2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 14:18:50.587145 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.587050    2584 factory.go:103] Registering Raw factory
Jul 28 14:18:50.587145 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.587069    2584 manager.go:1196] Started watching for new ooms in manager
Jul 28 14:18:50.587602 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.587585    2584 manager.go:349] Starting recovery of all containers
Jul 28 14:18:50.595181 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.594951    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 14:18:50.600127 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:50.600089    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-66.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 14:18:50.600276 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.600146    2584 manager.go:354] Recovery completed
Jul 28 14:18:50.600387 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:50.600340    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 14:18:50.605342 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.605323    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 14:18:50.605443 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.605425    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z9qwt"
Jul 28 14:18:50.611238 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.611218    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 14:18:50.611322 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.611269    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 14:18:50.611322 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.611283    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Jul 28 14:18:50.612448 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.612430    2584 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 14:18:50.612448 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.612443    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 14:18:50.612575 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.612446    2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 14:18:50.612575 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.612523    2584 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 14:18:50.613727 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.613707    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 14:18:50.613825 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.613750    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 14:18:50.613825 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.613768    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Jul 28 14:18:50.614947 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.614571    2584 policy_none.go:49] "None policy: Start"
Jul 28 14:18:50.614947 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.614594    2584 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 14:18:50.614947 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.614611    2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 14:18:50.615859 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.615842    2584 policy_none.go:47] "Start"
Jul 28 14:18:50.618338 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.618320    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z9qwt"
Jul 28 14:18:50.622395 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:50.622310    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-66.ec2.internal.18c679e5e5f1273d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-66.ec2.internal,UID:ip-10-0-130-66.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-66.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-66.ec2.internal,},FirstTimestamp:2026-07-28 14:18:50.611238717 +0000 UTC m=+0.412780936,LastTimestamp:2026-07-28 14:18:50.611238717 +0000 UTC m=+0.412780936,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-66.ec2.internal,}"
Jul 28 14:18:50.624614 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:50.624547    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-66.ec2.internal.18c679e5e5f1b7d7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-66.ec2.internal,UID:ip-10-0-130-66.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-66.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-66.ec2.internal,},FirstTimestamp:2026-07-28 14:18:50.611275735 +0000 UTC m=+0.412817944,LastTimestamp:2026-07-28 14:18:50.611275735 +0000 UTC m=+0.412817944,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-66.ec2.internal,}"
Jul 28 14:18:50.657853 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.657830    2584 manager.go:337] "Starting Device Plugin manager"
Jul 28 14:18:50.657935 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:50.657878    2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 14:18:50.657935 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.657892    2584 server.go:87] "Starting device plugin registration server"
Jul 28 14:18:50.658249 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.658235    2584 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 14:18:50.658320 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.658250    2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 14:18:50.658361 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.658334    2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 14:18:50.658430 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.658415    2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 14:18:50.658430 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.658429    2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 14:18:50.659141 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:50.659120    2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 14:18:50.659247 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:50.659183    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 28 14:18:50.759342 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.759304    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 14:18:50.761018 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.760993    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 14:18:50.761193 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.761034    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 14:18:50.761193 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.761052    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Jul 28 14:18:50.761193 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.761091    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-66.ec2.internal"
Jul 28 14:18:50.771017 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.770989    2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-66.ec2.internal"
Jul 28 14:18:50.771134 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:50.771025    2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-66.ec2.internal\": node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 28 14:18:50.789758 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.789733    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 14:18:50.789936 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.789777    2584 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 14:18:50.789936 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.789799    2584 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 14:18:50.789936 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:50.789837    2584 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 14:18:50.792101 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.792077    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 14:18:50.792919 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:50.792899    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 28 14:18:50.890736 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.890676    2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"]
Jul 28 14:18:50.890883 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.890784    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 14:18:50.891829 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.891810    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 14:18:50.891938 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.891845    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 14:18:50.891938 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.891859    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Jul 28 14:18:50.893006 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:50.892987    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 28 14:18:50.893412 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.893399    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 14:18:50.893549 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.893533    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"
Jul 28 14:18:50.893608 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.893571    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 14:18:50.894159 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.894143    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 14:18:50.894272 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.894176    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 14:18:50.894272 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.894189    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 14:18:50.894272 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.894201    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 14:18:50.894272 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.894206    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Jul 28 14:18:50.894272 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.894216    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Jul 28 14:18:50.895398 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.895384    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Jul 28 14:18:50.895446 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.895411    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 14:18:50.896129 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.896114    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 14:18:50.896210 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.896146    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 14:18:50.896210 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.896162    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Jul 28 14:18:50.925197 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:50.925160    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-66.ec2.internal\" not found" node="ip-10-0-130-66.ec2.internal"
Jul 28 14:18:50.929987 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:50.929971    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-66.ec2.internal\" not found" node="ip-10-0-130-66.ec2.internal"
Jul 28 14:18:50.989152 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.989119    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f675f063811c81df70fa063df9f70b5b-config\") pod \"kube-apiserver-proxy-ip-10-0-130-66.ec2.internal\" (UID: \"f675f063811c81df70fa063df9f70b5b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"
Jul 28 14:18:50.989152 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.989153    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7e6933567c76424165705d8ba7f1f8be-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal\" (UID: \"7e6933567c76424165705d8ba7f1f8be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Jul 28 14:18:50.989354 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:50.989175    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e6933567c76424165705d8ba7f1f8be-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal\" (UID: \"7e6933567c76424165705d8ba7f1f8be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Jul 28 14:18:50.993957 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:50.993935    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 28 14:18:51.089783 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.089686    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f675f063811c81df70fa063df9f70b5b-config\") pod \"kube-apiserver-proxy-ip-10-0-130-66.ec2.internal\" (UID: \"f675f063811c81df70fa063df9f70b5b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"
Jul 28 14:18:51.089783 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.089729    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7e6933567c76424165705d8ba7f1f8be-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal\" (UID: \"7e6933567c76424165705d8ba7f1f8be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Jul 28 14:18:51.089783 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.089754    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e6933567c76424165705d8ba7f1f8be-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal\" (UID: \"7e6933567c76424165705d8ba7f1f8be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Jul 28 14:18:51.090010 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.089798    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e6933567c76424165705d8ba7f1f8be-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal\" (UID: \"7e6933567c76424165705d8ba7f1f8be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Jul 28 14:18:51.090010 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.089801    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f675f063811c81df70fa063df9f70b5b-config\") pod \"kube-apiserver-proxy-ip-10-0-130-66.ec2.internal\" (UID: \"f675f063811c81df70fa063df9f70b5b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"
Jul 28 14:18:51.090010 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.089801    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7e6933567c76424165705d8ba7f1f8be-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal\" (UID: \"7e6933567c76424165705d8ba7f1f8be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Jul 28 14:18:51.094890 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:51.094861    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 28 14:18:51.195974 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:51.195938    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 28 14:18:51.229329 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.229287    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"
Jul 28 14:18:51.233440 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.233203    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Jul 28 14:18:51.296572 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:51.296541    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 28 14:18:51.397723 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:51.397651    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 28 14:18:51.426086 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.426054    2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 14:18:51.486795 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.486755    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"
Jul 28 14:18:51.500160 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.500133    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 14:18:51.502149 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.502131    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Jul 28 14:18:51.514109 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.514067    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 14:18:51.521750 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.521729    2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 14:18:51.521895 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.521873    2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 14:18:51.521942 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.521886    2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 14:18:51.564153 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.564120    2584 apiserver.go:52] "Watching apiserver"
Jul 28 14:18:51.577375 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.577339    2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 14:18:51.584751 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.584722    2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 14:18:51.585795 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.585754    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn","openshift-cluster-node-tuning-operator/tuned-df54l","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal","openshift-multus/multus-additional-cni-plugins-p789p","openshift-network-operator/iptables-alerter-j746c","openshift-ovn-kubernetes/ovnkube-node-lr6gc","kube-system/konnectivity-agent-h2ddz","openshift-dns/node-resolver-k66dk","openshift-image-registry/node-ca-jldlr","openshift-multus/multus-rsxdj","openshift-multus/network-metrics-daemon-fx9qn","openshift-network-diagnostics/network-check-target-lcn77","kube-system/global-pull-secret-syncer-d7fhl"]
Jul 28 14:18:51.585953 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.585774    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 14:18:51.587616 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.587598    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.588835 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.588817    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.589944 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.589923    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.590467 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.590430    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 14:18:51.590467 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.590448    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 14:18:51.590467 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.590460    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ck7g4\""
Jul 28 14:18:51.590666 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.590500    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 14:18:51.591083 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.591069    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j746c"
Jul 28 14:18:51.591325 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.591303    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 14:18:51.591552 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.591531    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 14:18:51.591716 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.591694    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-k86n8\""
Jul 28 14:18:51.592458 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.592440    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 14:18:51.592760 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.592738    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5rndf\""
Jul 28 14:18:51.592854 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.592792    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 14:18:51.592854 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.592803    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.593367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.593345    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 14:18:51.593367 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.593362    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 14:18:51.593514 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.593369    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 14:18:51.593676 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.593661    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-h5frk\""
Jul 28 14:18:51.594094 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.594079    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 14:18:51.594183 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.594159    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 14:18:51.594311 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.594295    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 14:18:51.594379 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.594365    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h2ddz"
Jul 28 14:18:51.594522 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.594498    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c54a401c-f28b-49df-9323-26d3ee747317-etc-selinux\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.594594 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.594539    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c54a401c-f28b-49df-9323-26d3ee747317-registration-dir\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.594594 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.594576    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c54a401c-f28b-49df-9323-26d3ee747317-device-dir\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.594689 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.594604    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c54a401c-f28b-49df-9323-26d3ee747317-sys-fs\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.594689 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.594637    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c55jp\" (UniqueName: \"kubernetes.io/projected/c54a401c-f28b-49df-9323-26d3ee747317-kube-api-access-c55jp\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.594689 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.594659    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c54a401c-f28b-49df-9323-26d3ee747317-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.594689 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.594674    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c54a401c-f28b-49df-9323-26d3ee747317-socket-dir\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.595438 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.595413    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 14:18:51.595535 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.595456    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 14:18:51.595535 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.595481    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hkw49\""
Jul 28 14:18:51.595535 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.595483    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 14:18:51.595701 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.595583    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 14:18:51.595842 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.595824    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 14:18:51.596068 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.596050    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k66dk"
Jul 28 14:18:51.596274 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.596254    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 14:18:51.596703 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.596688    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 14:18:51.596778 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.596722    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 14:18:51.596778 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.596741    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hjvwv\""
Jul 28 14:18:51.597723 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.597703    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jldlr"
Jul 28 14:18:51.598559 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.598539    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sl79r\""
Jul 28 14:18:51.598645 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.598536    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 14:18:51.598887 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.598869    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 14:18:51.599288 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.599269    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 14:18:51.599517 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.599499    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.600052 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.600036    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 14:18:51.600224 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.600210    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 14:18:51.600366 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.600352    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 14:18:51.600366 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.600363    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bpmqf\""
Jul 28 14:18:51.601339 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.601321    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:18:51.601431 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:51.601405    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx9qn" podUID="9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c"
Jul 28 14:18:51.601865 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.601848    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 14:18:51.601922 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.601851    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mqrkn\""
Jul 28 14:18:51.603032 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.603014    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:18:51.603111 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:51.603069    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lcn77" podUID="dbe349f8-37a2-49ce-b92c-6100612b24b6"
Jul 28 14:18:51.604352 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.604334    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:18:51.604410 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:51.604392    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7fhl" podUID="9ffc9cda-254d-4825-bb1a-1872331a8cc9"
Jul 28 14:18:51.624890 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.624833    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 14:13:50 +0000 UTC" deadline="2028-03-28 07:33:00.974905754 +0000 UTC"
Jul 28 14:18:51.624890 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.624891    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14609h14m9.350019349s"
Jul 28 14:18:51.626299 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.626280    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dv9rs"
Jul 28 14:18:51.636603 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.636582    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dv9rs"
Jul 28 14:18:51.688044 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.688021    2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 14:18:51.694969 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.694936    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c54a401c-f28b-49df-9323-26d3ee747317-registration-dir\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.694969 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.694972    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c54a401c-f28b-49df-9323-26d3ee747317-device-dir\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.695162 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.694994    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2dfdeb6-5821-484f-8548-5d305d57a790-system-cni-dir\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.695162 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695014    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs\") pod \"network-metrics-daemon-fx9qn\" (UID: \"9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c\") " pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:18:51.695162 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695039    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7ddn\" (UniqueName: \"kubernetes.io/projected/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-kube-api-access-n7ddn\") pod \"network-metrics-daemon-fx9qn\" (UID: \"9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c\") " pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:18:51.695162 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695079    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c54a401c-f28b-49df-9323-26d3ee747317-registration-dir\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.695162 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695082    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c54a401c-f28b-49df-9323-26d3ee747317-device-dir\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.695162 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695150    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-env-overrides\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.695507 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695204    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-ovn-node-metrics-cert\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.695507 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695236    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-cni-netd\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.695507 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695282    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d3c1ce2b-ee28-4657-81d3-a2f28736fab4-hosts-file\") pod \"node-resolver-k66dk\" (UID: \"d3c1ce2b-ee28-4657-81d3-a2f28736fab4\") " pod="openshift-dns/node-resolver-k66dk"
Jul 28 14:18:51.695507 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695310    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-multus-cni-dir\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.695507 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695326    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-host-run-netns\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.695507 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695376    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-log-socket\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.695507 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695409    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c54a401c-f28b-49df-9323-26d3ee747317-sys-fs\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.695507 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695459    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-sysctl-conf\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.695507 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695503    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-host\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.695846 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695553    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b232405a-e46c-4944-ac1a-954bfef6190e-host-slash\") pod \"iptables-alerter-j746c\" (UID: \"b232405a-e46c-4944-ac1a-954bfef6190e\") " pod="openshift-network-operator/iptables-alerter-j746c"
Jul 28 14:18:51.695846 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695570    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c54a401c-f28b-49df-9323-26d3ee747317-sys-fs\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.695846 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695601    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-run-ovn\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.695846 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695638    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfnj4\" (UniqueName: \"kubernetes.io/projected/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-kube-api-access-pfnj4\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.695846 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695690    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-system-cni-dir\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.695846 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695725    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-run-systemd\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.695846 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695755    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9ffc9cda-254d-4825-bb1a-1872331a8cc9-kubelet-config\") pod \"global-pull-secret-syncer-d7fhl\" (UID: \"9ffc9cda-254d-4825-bb1a-1872331a8cc9\") " pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:18:51.695846 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695773    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9ffc9cda-254d-4825-bb1a-1872331a8cc9-dbus\") pod \"global-pull-secret-syncer-d7fhl\" (UID: \"9ffc9cda-254d-4825-bb1a-1872331a8cc9\") " pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:18:51.695846 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695793    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-sys\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.695846 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695819    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3c1ce2b-ee28-4657-81d3-a2f28736fab4-tmp-dir\") pod \"node-resolver-k66dk\" (UID: \"d3c1ce2b-ee28-4657-81d3-a2f28736fab4\") " pod="openshift-dns/node-resolver-k66dk"
Jul 28 14:18:51.695846 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695850    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-cnibin\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.696342 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695867    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-host-run-k8s-cni-cncf-io\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.696342 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695889    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-multus-conf-dir\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.696342 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695910    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-lib-modules\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.696342 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695936    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-tuned\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.696342 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695955    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpzvm\" (UniqueName: \"kubernetes.io/projected/26358014-f39f-48a3-94df-fcc8c7ba60dc-kube-api-access-lpzvm\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.696342 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.695979    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c2dfdeb6-5821-484f-8548-5d305d57a790-cni-binary-copy\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.696342 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696014    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-kubelet\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.696342 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696057    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-systemd-units\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.696342 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696098    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-slash\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.696342 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696216    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-etc-openvswitch\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.696342 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696264    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-sysctl-d\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.696342 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696292    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-ovnkube-config\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.696342 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696322    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdr7g\" (UniqueName: \"kubernetes.io/projected/d3c1ce2b-ee28-4657-81d3-a2f28736fab4-kube-api-access-xdr7g\") pod \"node-resolver-k66dk\" (UID: \"d3c1ce2b-ee28-4657-81d3-a2f28736fab4\") " pod="openshift-dns/node-resolver-k66dk"
Jul 28 14:18:51.696892 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696391    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c55jp\" (UniqueName: \"kubernetes.io/projected/c54a401c-f28b-49df-9323-26d3ee747317-kube-api-access-c55jp\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.696892 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696563    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-run-openvswitch\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.696892 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696592    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-cni-bin\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.696892 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696618    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-multus-socket-dir-parent\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.696892 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696643    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-host-var-lib-cni-multus\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.696892 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696667    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-hostroot\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.696892 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696691    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-etc-kubernetes\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.696892 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696718    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26358014-f39f-48a3-94df-fcc8c7ba60dc-tmp\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.696892 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696741    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-os-release\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.696892 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696781    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-host-var-lib-cni-bin\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.696892 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696846    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-kubernetes\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.697281 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696875    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c2dfdeb6-5821-484f-8548-5d305d57a790-cnibin\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.697281 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696930    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d943e044-62ab-4f2c-a9a6-10fc30a01aca-konnectivity-ca\") pod \"konnectivity-agent-h2ddz\" (UID: \"d943e044-62ab-4f2c-a9a6-10fc30a01aca\") " pod="kube-system/konnectivity-agent-h2ddz"
Jul 28 14:18:51.697281 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.696989    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-var-lib-openvswitch\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.697281 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697020    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-ovnkube-script-lib\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.697281 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697051    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c54a401c-f28b-49df-9323-26d3ee747317-socket-dir\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.697281 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697082    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c54a401c-f28b-49df-9323-26d3ee747317-etc-selinux\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.697281 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697109    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d943e044-62ab-4f2c-a9a6-10fc30a01aca-agent-certs\") pod \"konnectivity-agent-h2ddz\" (UID: \"d943e044-62ab-4f2c-a9a6-10fc30a01aca\") " pod="kube-system/konnectivity-agent-h2ddz"
Jul 28 14:18:51.697281 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697162    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d78s\" (UniqueName: \"kubernetes.io/projected/e1374453-20fd-4a88-bef4-82e998969586-kube-api-access-7d78s\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.697281 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697187    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-systemd\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.697281 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697217    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.697281 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697250    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-host-run-multus-certs\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.697281 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697277    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-var-lib-kubelet\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.697705 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697325    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c54a401c-f28b-49df-9323-26d3ee747317-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.697705 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697357    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-run\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.697705 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697386    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7ccx\" (UniqueName: \"kubernetes.io/projected/b232405a-e46c-4944-ac1a-954bfef6190e-kube-api-access-x7ccx\") pod \"iptables-alerter-j746c\" (UID: \"b232405a-e46c-4944-ac1a-954bfef6190e\") " pod="openshift-network-operator/iptables-alerter-j746c"
Jul 28 14:18:51.697705 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697416    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c2dfdeb6-5821-484f-8548-5d305d57a790-os-release\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.697705 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697445    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwbtj\" (UniqueName: \"kubernetes.io/projected/c2dfdeb6-5821-484f-8548-5d305d57a790-kube-api-access-kwbtj\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.697705 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697471    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c54a401c-f28b-49df-9323-26d3ee747317-etc-selinux\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.697705 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697474    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-node-log\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.697705 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697579    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c54a401c-f28b-49df-9323-26d3ee747317-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.697705 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697626    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-run-ovn-kubernetes\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.697705 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697655    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e1374453-20fd-4a88-bef4-82e998969586-cni-binary-copy\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.697705 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697685    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b232405a-e46c-4944-ac1a-954bfef6190e-iptables-alerter-script\") pod \"iptables-alerter-j746c\" (UID: \"b232405a-e46c-4944-ac1a-954bfef6190e\") " pod="openshift-network-operator/iptables-alerter-j746c"
Jul 28 14:18:51.697705 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697691    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c54a401c-f28b-49df-9323-26d3ee747317-socket-dir\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.698103 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697710    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-host-var-lib-kubelet\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.698103 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697736    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-modprobe-d\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.698103 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697773    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-run-netns\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.698103 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697807    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e1374453-20fd-4a88-bef4-82e998969586-multus-daemon-config\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.698103 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697836    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlwv6\" (UniqueName: \"kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6\") pod \"network-check-target-lcn77\" (UID: \"dbe349f8-37a2-49ce-b92c-6100612b24b6\") " pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:18:51.698103 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697875    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret\") pod \"global-pull-secret-syncer-d7fhl\" (UID: \"9ffc9cda-254d-4825-bb1a-1872331a8cc9\") " pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:18:51.698103 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697908    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-sysconfig\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.698103 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697939    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c2dfdeb6-5821-484f-8548-5d305d57a790-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.698103 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.697968    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c2dfdeb6-5821-484f-8548-5d305d57a790-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.698103 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.698001    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c2dfdeb6-5821-484f-8548-5d305d57a790-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.698103 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.698028    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1c2b854d-bbb4-4b63-b880-f71b7df99fdd-host\") pod \"node-ca-jldlr\" (UID: \"1c2b854d-bbb4-4b63-b880-f71b7df99fdd\") " pod="openshift-image-registry/node-ca-jldlr"
Jul 28 14:18:51.698103 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.698061    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1c2b854d-bbb4-4b63-b880-f71b7df99fdd-serviceca\") pod \"node-ca-jldlr\" (UID: \"1c2b854d-bbb4-4b63-b880-f71b7df99fdd\") " pod="openshift-image-registry/node-ca-jldlr"
Jul 28 14:18:51.698103 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.698094    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wssqd\" (UniqueName: \"kubernetes.io/projected/1c2b854d-bbb4-4b63-b880-f71b7df99fdd-kube-api-access-wssqd\") pod \"node-ca-jldlr\" (UID: \"1c2b854d-bbb4-4b63-b880-f71b7df99fdd\") " pod="openshift-image-registry/node-ca-jldlr"
Jul 28 14:18:51.705678 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.705641    2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 14:18:51.709232 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.709200    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c55jp\" (UniqueName: \"kubernetes.io/projected/c54a401c-f28b-49df-9323-26d3ee747317-kube-api-access-c55jp\") pod \"aws-ebs-csi-driver-node-7fhwn\" (UID: \"c54a401c-f28b-49df-9323-26d3ee747317\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.789759 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:51.789727    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e6933567c76424165705d8ba7f1f8be.slice/crio-76fc24ab27e13dbdaee4e9152487c9f7883fbc94e91e6b1b3c3d4029e3c26627 WatchSource:0}: Error finding container 76fc24ab27e13dbdaee4e9152487c9f7883fbc94e91e6b1b3c3d4029e3c26627: Status 404 returned error can't find the container with id 76fc24ab27e13dbdaee4e9152487c9f7883fbc94e91e6b1b3c3d4029e3c26627
Jul 28 14:18:51.790117 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:51.790104    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf675f063811c81df70fa063df9f70b5b.slice/crio-d4e5e38c4b50e31c3f71e960df3686c71aa11e02accac1bf1c0b930f88aa0df4 WatchSource:0}: Error finding container d4e5e38c4b50e31c3f71e960df3686c71aa11e02accac1bf1c0b930f88aa0df4: Status 404 returned error can't find the container with id d4e5e38c4b50e31c3f71e960df3686c71aa11e02accac1bf1c0b930f88aa0df4
Jul 28 14:18:51.795764 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.795747    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 14:18:51.798687 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.798664    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-run-netns\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.798768 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.798696    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e1374453-20fd-4a88-bef4-82e998969586-multus-daemon-config\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.798768 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.798714    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlwv6\" (UniqueName: \"kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6\") pod \"network-check-target-lcn77\" (UID: \"dbe349f8-37a2-49ce-b92c-6100612b24b6\") " pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:18:51.798768 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.798732    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret\") pod \"global-pull-secret-syncer-d7fhl\" (UID: \"9ffc9cda-254d-4825-bb1a-1872331a8cc9\") " pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:18:51.798926 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.798780    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-run-netns\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.798926 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.798835    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-sysconfig\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.798926 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.798860    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c2dfdeb6-5821-484f-8548-5d305d57a790-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.798926 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.798887    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c2dfdeb6-5821-484f-8548-5d305d57a790-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.798926 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.798917    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c2dfdeb6-5821-484f-8548-5d305d57a790-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.799201 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:51.798988    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:51.799201 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799012    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1c2b854d-bbb4-4b63-b880-f71b7df99fdd-host\") pod \"node-ca-jldlr\" (UID: \"1c2b854d-bbb4-4b63-b880-f71b7df99fdd\") " pod="openshift-image-registry/node-ca-jldlr"
Jul 28 14:18:51.799201 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799066    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1c2b854d-bbb4-4b63-b880-f71b7df99fdd-host\") pod \"node-ca-jldlr\" (UID: \"1c2b854d-bbb4-4b63-b880-f71b7df99fdd\") " pod="openshift-image-registry/node-ca-jldlr"
Jul 28 14:18:51.799201 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:51.799072    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret podName:9ffc9cda-254d-4825-bb1a-1872331a8cc9 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:52.299031 +0000 UTC m=+2.100573214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret") pod "global-pull-secret-syncer-d7fhl" (UID: "9ffc9cda-254d-4825-bb1a-1872331a8cc9") : object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:51.799201 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.798979    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-sysconfig\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.799201 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799093    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1c2b854d-bbb4-4b63-b880-f71b7df99fdd-serviceca\") pod \"node-ca-jldlr\" (UID: \"1c2b854d-bbb4-4b63-b880-f71b7df99fdd\") " pod="openshift-image-registry/node-ca-jldlr"
Jul 28 14:18:51.799201 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799108    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c2dfdeb6-5821-484f-8548-5d305d57a790-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.799201 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799178    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wssqd\" (UniqueName: \"kubernetes.io/projected/1c2b854d-bbb4-4b63-b880-f71b7df99fdd-kube-api-access-wssqd\") pod \"node-ca-jldlr\" (UID: \"1c2b854d-bbb4-4b63-b880-f71b7df99fdd\") " pod="openshift-image-registry/node-ca-jldlr"
Jul 28 14:18:51.799663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799234    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2dfdeb6-5821-484f-8548-5d305d57a790-system-cni-dir\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.799663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799283    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs\") pod \"network-metrics-daemon-fx9qn\" (UID: \"9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c\") " pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:18:51.799663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799312    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7ddn\" (UniqueName: \"kubernetes.io/projected/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-kube-api-access-n7ddn\") pod \"network-metrics-daemon-fx9qn\" (UID: \"9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c\") " pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:18:51.799663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799341    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-env-overrides\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.799663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799366    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-ovn-node-metrics-cert\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.799663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799392    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-cni-netd\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.799663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799424    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d3c1ce2b-ee28-4657-81d3-a2f28736fab4-hosts-file\") pod \"node-resolver-k66dk\" (UID: \"d3c1ce2b-ee28-4657-81d3-a2f28736fab4\") " pod="openshift-dns/node-resolver-k66dk"
Jul 28 14:18:51.799663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799451    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-multus-cni-dir\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.799663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799475    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-host-run-netns\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.799663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799504    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-log-socket\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.799663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799534    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e1374453-20fd-4a88-bef4-82e998969586-multus-daemon-config\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.799663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799551    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-sysctl-conf\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.799663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799579    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-host\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.799663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799605    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b232405a-e46c-4944-ac1a-954bfef6190e-host-slash\") pod \"iptables-alerter-j746c\" (UID: \"b232405a-e46c-4944-ac1a-954bfef6190e\") " pod="openshift-network-operator/iptables-alerter-j746c"
Jul 28 14:18:51.799663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799614    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c2dfdeb6-5821-484f-8548-5d305d57a790-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.799663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799630    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d3c1ce2b-ee28-4657-81d3-a2f28736fab4-hosts-file\") pod \"node-resolver-k66dk\" (UID: \"d3c1ce2b-ee28-4657-81d3-a2f28736fab4\") " pod="openshift-dns/node-resolver-k66dk"
Jul 28 14:18:51.799663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799634    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c2dfdeb6-5821-484f-8548-5d305d57a790-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.800590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799655    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2dfdeb6-5821-484f-8548-5d305d57a790-system-cni-dir\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.800590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799672    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b232405a-e46c-4944-ac1a-954bfef6190e-host-slash\") pod \"iptables-alerter-j746c\" (UID: \"b232405a-e46c-4944-ac1a-954bfef6190e\") " pod="openshift-network-operator/iptables-alerter-j746c"
Jul 28 14:18:51.800590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799680    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-multus-cni-dir\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.800590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799704    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-run-ovn\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.800590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799736    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfnj4\" (UniqueName: \"kubernetes.io/projected/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-kube-api-access-pfnj4\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.800590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799750    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1c2b854d-bbb4-4b63-b880-f71b7df99fdd-serviceca\") pod \"node-ca-jldlr\" (UID: \"1c2b854d-bbb4-4b63-b880-f71b7df99fdd\") " pod="openshift-image-registry/node-ca-jldlr"
Jul 28 14:18:51.800590 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:51.799757    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:18:51.800590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799763    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-system-cni-dir\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.800590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799789    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-run-systemd\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.800590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799794    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-sysctl-conf\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.800590 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:51.799821    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs podName:9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c nodeName:}" failed. No retries permitted until 2026-07-28 14:18:52.2997861 +0000 UTC m=+2.101328302 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs") pod "network-metrics-daemon-fx9qn" (UID: "9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:18:51.800590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799821    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-host\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.800590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799842    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-run-systemd\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.800590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799836    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-run-ovn\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.800590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799878    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-cni-netd\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.800590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799887    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-host-run-netns\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.800590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799920    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9ffc9cda-254d-4825-bb1a-1872331a8cc9-kubelet-config\") pod \"global-pull-secret-syncer-d7fhl\" (UID: \"9ffc9cda-254d-4825-bb1a-1872331a8cc9\") " pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:18:51.800590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799928    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-system-cni-dir\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.801632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799948    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9ffc9cda-254d-4825-bb1a-1872331a8cc9-dbus\") pod \"global-pull-secret-syncer-d7fhl\" (UID: \"9ffc9cda-254d-4825-bb1a-1872331a8cc9\") " pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:18:51.801632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799976    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-sys\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.801632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.799979    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9ffc9cda-254d-4825-bb1a-1872331a8cc9-kubelet-config\") pod \"global-pull-secret-syncer-d7fhl\" (UID: \"9ffc9cda-254d-4825-bb1a-1872331a8cc9\") " pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:18:51.801632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800002    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3c1ce2b-ee28-4657-81d3-a2f28736fab4-tmp-dir\") pod \"node-resolver-k66dk\" (UID: \"d3c1ce2b-ee28-4657-81d3-a2f28736fab4\") " pod="openshift-dns/node-resolver-k66dk"
Jul 28 14:18:51.801632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800028    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-cnibin\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.801632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800041    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-log-socket\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.801632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800051    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-host-run-k8s-cni-cncf-io\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.801632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800082    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-multus-conf-dir\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.801632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800112    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-lib-modules\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.801632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800117    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-env-overrides\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.801632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800136    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-tuned\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.801632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800122    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9ffc9cda-254d-4825-bb1a-1872331a8cc9-dbus\") pod \"global-pull-secret-syncer-d7fhl\" (UID: \"9ffc9cda-254d-4825-bb1a-1872331a8cc9\") " pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:18:51.801632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800161    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpzvm\" (UniqueName: \"kubernetes.io/projected/26358014-f39f-48a3-94df-fcc8c7ba60dc-kube-api-access-lpzvm\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.801632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800183    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-sys\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.801632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800243    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-host-run-k8s-cni-cncf-io\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.801632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800286    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-lib-modules\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.801632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800300    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-cnibin\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.801632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800323    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c2dfdeb6-5821-484f-8548-5d305d57a790-cni-binary-copy\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.802584 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800358    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3c1ce2b-ee28-4657-81d3-a2f28736fab4-tmp-dir\") pod \"node-resolver-k66dk\" (UID: \"d3c1ce2b-ee28-4657-81d3-a2f28736fab4\") " pod="openshift-dns/node-resolver-k66dk"
Jul 28 14:18:51.802584 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800354    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-kubelet\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.802584 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800397    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-systemd-units\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.802584 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800422    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-slash\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.802584 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800449    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-etc-openvswitch\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.802584 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800476    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-sysctl-d\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.802584 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800503    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-ovnkube-config\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.802584 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800538    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdr7g\" (UniqueName: \"kubernetes.io/projected/d3c1ce2b-ee28-4657-81d3-a2f28736fab4-kube-api-access-xdr7g\") pod \"node-resolver-k66dk\" (UID: \"d3c1ce2b-ee28-4657-81d3-a2f28736fab4\") " pod="openshift-dns/node-resolver-k66dk"
Jul 28 14:18:51.802584 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800569    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-run-openvswitch\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.802584 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800574    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-slash\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.802584 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800605    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-cni-bin\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.802584 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800621    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-kubelet\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.802584 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800633    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-multus-socket-dir-parent\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.802584 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800664    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-host-var-lib-cni-multus\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.802584 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800691    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-hostroot\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.802584 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800701    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-multus-conf-dir\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.802584 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800717    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-etc-kubernetes\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.802584 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800748    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-run-openvswitch\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.803472 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800752    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26358014-f39f-48a3-94df-fcc8c7ba60dc-tmp\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.803472 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800666    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-systemd-units\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.803472 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800790    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-os-release\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.803472 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800841    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c2dfdeb6-5821-484f-8548-5d305d57a790-cni-binary-copy\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.803472 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800878    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-host-var-lib-cni-bin\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.803472 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800887    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-hostroot\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.803472 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800891    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-cni-bin\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.803472 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800847    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-etc-kubernetes\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.803472 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800907    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-kubernetes\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.803472 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800847    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-os-release\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.803472 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800940    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c2dfdeb6-5821-484f-8548-5d305d57a790-cnibin\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.803472 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800962    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-kubernetes\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.803472 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800971    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-host-var-lib-cni-multus\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.803472 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801059    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c2dfdeb6-5821-484f-8548-5d305d57a790-cnibin\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.803472 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801072    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-sysctl-d\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.803472 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801112    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-etc-openvswitch\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.803472 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801122    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-multus-socket-dir-parent\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.803472 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800969    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d943e044-62ab-4f2c-a9a6-10fc30a01aca-konnectivity-ca\") pod \"konnectivity-agent-h2ddz\" (UID: \"d943e044-62ab-4f2c-a9a6-10fc30a01aca\") " pod="kube-system/konnectivity-agent-h2ddz"
Jul 28 14:18:51.804122 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801163    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-var-lib-openvswitch\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.804122 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.800941    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-host-var-lib-cni-bin\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.804122 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801209    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-ovnkube-script-lib\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.804122 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801236    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d943e044-62ab-4f2c-a9a6-10fc30a01aca-agent-certs\") pod \"konnectivity-agent-h2ddz\" (UID: \"d943e044-62ab-4f2c-a9a6-10fc30a01aca\") " pod="kube-system/konnectivity-agent-h2ddz"
Jul 28 14:18:51.804122 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801258    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d78s\" (UniqueName: \"kubernetes.io/projected/e1374453-20fd-4a88-bef4-82e998969586-kube-api-access-7d78s\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.804122 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801274    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-systemd\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.804122 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801291    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.804122 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801308    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-host-run-multus-certs\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.804122 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801329    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-ovnkube-config\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.804122 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801323    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-var-lib-kubelet\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.804122 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801383    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-run\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.804122 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801405    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7ccx\" (UniqueName: \"kubernetes.io/projected/b232405a-e46c-4944-ac1a-954bfef6190e-kube-api-access-x7ccx\") pod \"iptables-alerter-j746c\" (UID: \"b232405a-e46c-4944-ac1a-954bfef6190e\") " pod="openshift-network-operator/iptables-alerter-j746c"
Jul 28 14:18:51.804122 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801423    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c2dfdeb6-5821-484f-8548-5d305d57a790-os-release\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.804122 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801444    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwbtj\" (UniqueName: \"kubernetes.io/projected/c2dfdeb6-5821-484f-8548-5d305d57a790-kube-api-access-kwbtj\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.804122 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801460    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-node-log\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.804122 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801476    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-run-ovn-kubernetes\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.804122 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801495    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e1374453-20fd-4a88-bef4-82e998969586-cni-binary-copy\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.804785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801511    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b232405a-e46c-4944-ac1a-954bfef6190e-iptables-alerter-script\") pod \"iptables-alerter-j746c\" (UID: \"b232405a-e46c-4944-ac1a-954bfef6190e\") " pod="openshift-network-operator/iptables-alerter-j746c"
Jul 28 14:18:51.804785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801526    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-host-var-lib-kubelet\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.804785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801551    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-modprobe-d\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.804785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801660    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-modprobe-d\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.804785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801697    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-systemd\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.804785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801712    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-ovnkube-script-lib\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.804785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801722    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.804785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801750    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-var-lib-kubelet\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.804785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801765    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-host-run-multus-certs\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.804785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801788    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26358014-f39f-48a3-94df-fcc8c7ba60dc-run\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.804785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801795    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-node-log\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.804785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801828    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-host-run-ovn-kubernetes\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.804785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.802041    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c2dfdeb6-5821-484f-8548-5d305d57a790-os-release\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.804785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.802170    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d943e044-62ab-4f2c-a9a6-10fc30a01aca-konnectivity-ca\") pod \"konnectivity-agent-h2ddz\" (UID: \"d943e044-62ab-4f2c-a9a6-10fc30a01aca\") " pod="kube-system/konnectivity-agent-h2ddz"
Jul 28 14:18:51.804785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.802298    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1374453-20fd-4a88-bef4-82e998969586-host-var-lib-kubelet\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.804785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.802307    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e1374453-20fd-4a88-bef4-82e998969586-cni-binary-copy\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.804785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.801238    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-var-lib-openvswitch\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.804785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.802644    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26358014-f39f-48a3-94df-fcc8c7ba60dc-etc-tuned\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.805333 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.802684    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-ovn-node-metrics-cert\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.805333 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.802698    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b232405a-e46c-4944-ac1a-954bfef6190e-iptables-alerter-script\") pod \"iptables-alerter-j746c\" (UID: \"b232405a-e46c-4944-ac1a-954bfef6190e\") " pod="openshift-network-operator/iptables-alerter-j746c"
Jul 28 14:18:51.805333 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.803411    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26358014-f39f-48a3-94df-fcc8c7ba60dc-tmp\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.805333 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.804149    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d943e044-62ab-4f2c-a9a6-10fc30a01aca-agent-certs\") pod \"konnectivity-agent-h2ddz\" (UID: \"d943e044-62ab-4f2c-a9a6-10fc30a01aca\") " pod="kube-system/konnectivity-agent-h2ddz"
Jul 28 14:18:51.813502 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:51.813479    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 14:18:51.813502 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:51.813504    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 14:18:51.813677 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:51.813518    2584 projected.go:196] Error preparing data for projected volume kube-api-access-vlwv6 for pod openshift-network-diagnostics/network-check-target-lcn77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:51.813677 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:51.813583    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6 podName:dbe349f8-37a2-49ce-b92c-6100612b24b6 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:52.31356517 +0000 UTC m=+2.115107379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vlwv6" (UniqueName: "kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6") pod "network-check-target-lcn77" (UID: "dbe349f8-37a2-49ce-b92c-6100612b24b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:51.816427 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.816399    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7ccx\" (UniqueName: \"kubernetes.io/projected/b232405a-e46c-4944-ac1a-954bfef6190e-kube-api-access-x7ccx\") pod \"iptables-alerter-j746c\" (UID: \"b232405a-e46c-4944-ac1a-954bfef6190e\") " pod="openshift-network-operator/iptables-alerter-j746c"
Jul 28 14:18:51.816559 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.816477    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d78s\" (UniqueName: \"kubernetes.io/projected/e1374453-20fd-4a88-bef4-82e998969586-kube-api-access-7d78s\") pod \"multus-rsxdj\" (UID: \"e1374453-20fd-4a88-bef4-82e998969586\") " pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:51.816631 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.816561    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpzvm\" (UniqueName: \"kubernetes.io/projected/26358014-f39f-48a3-94df-fcc8c7ba60dc-kube-api-access-lpzvm\") pod \"tuned-df54l\" (UID: \"26358014-f39f-48a3-94df-fcc8c7ba60dc\") " pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.816793 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.816760    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwbtj\" (UniqueName: \"kubernetes.io/projected/c2dfdeb6-5821-484f-8548-5d305d57a790-kube-api-access-kwbtj\") pod \"multus-additional-cni-plugins-p789p\" (UID: \"c2dfdeb6-5821-484f-8548-5d305d57a790\") " pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.816915 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.816898    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7ddn\" (UniqueName: \"kubernetes.io/projected/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-kube-api-access-n7ddn\") pod \"network-metrics-daemon-fx9qn\" (UID: \"9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c\") " pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:18:51.816973 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.816926    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfnj4\" (UniqueName: \"kubernetes.io/projected/8c78f0b5-33e8-40ec-994e-9c7d9448ac74-kube-api-access-pfnj4\") pod \"ovnkube-node-lr6gc\" (UID: \"8c78f0b5-33e8-40ec-994e-9c7d9448ac74\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.817331 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.817307    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdr7g\" (UniqueName: \"kubernetes.io/projected/d3c1ce2b-ee28-4657-81d3-a2f28736fab4-kube-api-access-xdr7g\") pod \"node-resolver-k66dk\" (UID: \"d3c1ce2b-ee28-4657-81d3-a2f28736fab4\") " pod="openshift-dns/node-resolver-k66dk"
Jul 28 14:18:51.817585 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.817567    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wssqd\" (UniqueName: \"kubernetes.io/projected/1c2b854d-bbb4-4b63-b880-f71b7df99fdd-kube-api-access-wssqd\") pod \"node-ca-jldlr\" (UID: \"1c2b854d-bbb4-4b63-b880-f71b7df99fdd\") " pod="openshift-image-registry/node-ca-jldlr"
Jul 28 14:18:51.916921 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.916832    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn"
Jul 28 14:18:51.926472 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:51.926441    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc54a401c_f28b_49df_9323_26d3ee747317.slice/crio-0d488896cc13a77699b38cac1852430f3608f3313d668a73afd4da6bde4a283c WatchSource:0}: Error finding container 0d488896cc13a77699b38cac1852430f3608f3313d668a73afd4da6bde4a283c: Status 404 returned error can't find the container with id 0d488896cc13a77699b38cac1852430f3608f3313d668a73afd4da6bde4a283c
Jul 28 14:18:51.940779 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.940758    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-df54l"
Jul 28 14:18:51.948667 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:51.948643    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26358014_f39f_48a3_94df_fcc8c7ba60dc.slice/crio-98ff271544d5be060cc807ac6486118ea13cb3dfb2231453e200a0339aab3370 WatchSource:0}: Error finding container 98ff271544d5be060cc807ac6486118ea13cb3dfb2231453e200a0339aab3370: Status 404 returned error can't find the container with id 98ff271544d5be060cc807ac6486118ea13cb3dfb2231453e200a0339aab3370
Jul 28 14:18:51.955186 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.955166    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p789p"
Jul 28 14:18:51.958804 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.958787    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j746c"
Jul 28 14:18:51.963070 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:51.963047    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2dfdeb6_5821_484f_8548_5d305d57a790.slice/crio-61ddf68545e3c0542cc3308235b7b74142b5e67a4134d96f113a695f8283c21f WatchSource:0}: Error finding container 61ddf68545e3c0542cc3308235b7b74142b5e67a4134d96f113a695f8283c21f: Status 404 returned error can't find the container with id 61ddf68545e3c0542cc3308235b7b74142b5e67a4134d96f113a695f8283c21f
Jul 28 14:18:51.967365 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:51.967344    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb232405a_e46c_4944_ac1a_954bfef6190e.slice/crio-3b315287dac77b1be7ee117a55ec60b99e66587d704b90923d7b52557ab4eb72 WatchSource:0}: Error finding container 3b315287dac77b1be7ee117a55ec60b99e66587d704b90923d7b52557ab4eb72: Status 404 returned error can't find the container with id 3b315287dac77b1be7ee117a55ec60b99e66587d704b90923d7b52557ab4eb72
Jul 28 14:18:51.967488 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.967476    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:18:51.976779 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:51.976753    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c78f0b5_33e8_40ec_994e_9c7d9448ac74.slice/crio-9f5be52cdf4d9ca045f8c8130f25ac0cc277a1a890a71bbb5b3e13d89e589ed9 WatchSource:0}: Error finding container 9f5be52cdf4d9ca045f8c8130f25ac0cc277a1a890a71bbb5b3e13d89e589ed9: Status 404 returned error can't find the container with id 9f5be52cdf4d9ca045f8c8130f25ac0cc277a1a890a71bbb5b3e13d89e589ed9
Jul 28 14:18:51.994700 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:51.994679    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h2ddz"
Jul 28 14:18:52.001935 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:52.001907    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd943e044_62ab_4f2c_a9a6_10fc30a01aca.slice/crio-edc39a3d7e65f73628ecaf24bc90fe9ce9e44d1ca5069aea1d240b65fd50942e WatchSource:0}: Error finding container edc39a3d7e65f73628ecaf24bc90fe9ce9e44d1ca5069aea1d240b65fd50942e: Status 404 returned error can't find the container with id edc39a3d7e65f73628ecaf24bc90fe9ce9e44d1ca5069aea1d240b65fd50942e
Jul 28 14:18:52.009818 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.009801    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k66dk"
Jul 28 14:18:52.017445 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:52.017421    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3c1ce2b_ee28_4657_81d3_a2f28736fab4.slice/crio-428ea400fbc02ec7a507d8e0a30061a640a96e81f8ad623544726a693099378a WatchSource:0}: Error finding container 428ea400fbc02ec7a507d8e0a30061a640a96e81f8ad623544726a693099378a: Status 404 returned error can't find the container with id 428ea400fbc02ec7a507d8e0a30061a640a96e81f8ad623544726a693099378a
Jul 28 14:18:52.033259 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.033238    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jldlr"
Jul 28 14:18:52.038974 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.038919    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rsxdj"
Jul 28 14:18:52.041422 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:52.041395    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c2b854d_bbb4_4b63_b880_f71b7df99fdd.slice/crio-8505079b063e9725f8c68cd6fc521106f0c28f519116a5ecbec9450ffaba11b3 WatchSource:0}: Error finding container 8505079b063e9725f8c68cd6fc521106f0c28f519116a5ecbec9450ffaba11b3: Status 404 returned error can't find the container with id 8505079b063e9725f8c68cd6fc521106f0c28f519116a5ecbec9450ffaba11b3
Jul 28 14:18:52.047053 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:18:52.047031    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1374453_20fd_4a88_bef4_82e998969586.slice/crio-0907e0cdc2c6d5b5450608023ad4b176e4f75b6a5770827be7a29e750729c656 WatchSource:0}: Error finding container 0907e0cdc2c6d5b5450608023ad4b176e4f75b6a5770827be7a29e750729c656: Status 404 returned error can't find the container with id 0907e0cdc2c6d5b5450608023ad4b176e4f75b6a5770827be7a29e750729c656
Jul 28 14:18:52.058062 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.058035    2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 14:18:52.304915 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.304799    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret\") pod \"global-pull-secret-syncer-d7fhl\" (UID: \"9ffc9cda-254d-4825-bb1a-1872331a8cc9\") " pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:18:52.304915 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.304866    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs\") pod \"network-metrics-daemon-fx9qn\" (UID: \"9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c\") " pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:18:52.305137 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:52.304947    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:52.305137 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:52.304961    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:18:52.305137 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:52.305016    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret podName:9ffc9cda-254d-4825-bb1a-1872331a8cc9 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:53.305000616 +0000 UTC m=+3.106542811 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret") pod "global-pull-secret-syncer-d7fhl" (UID: "9ffc9cda-254d-4825-bb1a-1872331a8cc9") : object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:52.305137 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:52.305031    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs podName:9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c nodeName:}" failed. No retries permitted until 2026-07-28 14:18:53.305024283 +0000 UTC m=+3.106566477 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs") pod "network-metrics-daemon-fx9qn" (UID: "9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:18:52.405320 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.405282    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlwv6\" (UniqueName: \"kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6\") pod \"network-check-target-lcn77\" (UID: \"dbe349f8-37a2-49ce-b92c-6100612b24b6\") " pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:18:52.405526 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:52.405501    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 14:18:52.405613 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:52.405538    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 14:18:52.405613 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:52.405555    2584 projected.go:196] Error preparing data for projected volume kube-api-access-vlwv6 for pod openshift-network-diagnostics/network-check-target-lcn77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:52.405723 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:52.405628    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6 podName:dbe349f8-37a2-49ce-b92c-6100612b24b6 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:53.40560565 +0000 UTC m=+3.207147862 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vlwv6" (UniqueName: "kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6") pod "network-check-target-lcn77" (UID: "dbe349f8-37a2-49ce-b92c-6100612b24b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:52.638796 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.638673    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 14:13:51 +0000 UTC" deadline="2028-05-07 02:14:41.024981686 +0000 UTC"
Jul 28 14:18:52.638796 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.638743    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15563h55m48.386242635s"
Jul 28 14:18:52.802800 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.802768    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:18:52.802994 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:52.802898    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7fhl" podUID="9ffc9cda-254d-4825-bb1a-1872331a8cc9"
Jul 28 14:18:52.803066 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.803036    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:18:52.803233 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:52.803194    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lcn77" podUID="dbe349f8-37a2-49ce-b92c-6100612b24b6"
Jul 28 14:18:52.803564 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.803484    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 14:18:52.805718 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.805636    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p789p" event={"ID":"c2dfdeb6-5821-484f-8548-5d305d57a790","Type":"ContainerStarted","Data":"61ddf68545e3c0542cc3308235b7b74142b5e67a4134d96f113a695f8283c21f"}
Jul 28 14:18:52.808839 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.808744    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-df54l" event={"ID":"26358014-f39f-48a3-94df-fcc8c7ba60dc","Type":"ContainerStarted","Data":"98ff271544d5be060cc807ac6486118ea13cb3dfb2231453e200a0339aab3370"}
Jul 28 14:18:52.813999 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.813970    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn" event={"ID":"c54a401c-f28b-49df-9323-26d3ee747317","Type":"ContainerStarted","Data":"0d488896cc13a77699b38cac1852430f3608f3313d668a73afd4da6bde4a283c"}
Jul 28 14:18:52.829856 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.829807    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal" event={"ID":"f675f063811c81df70fa063df9f70b5b","Type":"ContainerStarted","Data":"d4e5e38c4b50e31c3f71e960df3686c71aa11e02accac1bf1c0b930f88aa0df4"}
Jul 28 14:18:52.852959 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.852915    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jldlr" event={"ID":"1c2b854d-bbb4-4b63-b880-f71b7df99fdd","Type":"ContainerStarted","Data":"8505079b063e9725f8c68cd6fc521106f0c28f519116a5ecbec9450ffaba11b3"}
Jul 28 14:18:52.862483 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.862439    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc" event={"ID":"8c78f0b5-33e8-40ec-994e-9c7d9448ac74","Type":"ContainerStarted","Data":"9f5be52cdf4d9ca045f8c8130f25ac0cc277a1a890a71bbb5b3e13d89e589ed9"}
Jul 28 14:18:52.873905 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.873863    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j746c" event={"ID":"b232405a-e46c-4944-ac1a-954bfef6190e","Type":"ContainerStarted","Data":"3b315287dac77b1be7ee117a55ec60b99e66587d704b90923d7b52557ab4eb72"}
Jul 28 14:18:52.889728 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.889632    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal" event={"ID":"7e6933567c76424165705d8ba7f1f8be","Type":"ContainerStarted","Data":"76fc24ab27e13dbdaee4e9152487c9f7883fbc94e91e6b1b3c3d4029e3c26627"}
Jul 28 14:18:52.894983 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.894937    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rsxdj" event={"ID":"e1374453-20fd-4a88-bef4-82e998969586","Type":"ContainerStarted","Data":"0907e0cdc2c6d5b5450608023ad4b176e4f75b6a5770827be7a29e750729c656"}
Jul 28 14:18:52.909571 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.908560    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k66dk" event={"ID":"d3c1ce2b-ee28-4657-81d3-a2f28736fab4","Type":"ContainerStarted","Data":"428ea400fbc02ec7a507d8e0a30061a640a96e81f8ad623544726a693099378a"}
Jul 28 14:18:52.920685 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:52.920640    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h2ddz" event={"ID":"d943e044-62ab-4f2c-a9a6-10fc30a01aca","Type":"ContainerStarted","Data":"edc39a3d7e65f73628ecaf24bc90fe9ce9e44d1ca5069aea1d240b65fd50942e"}
Jul 28 14:18:53.314293 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:53.313920    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret\") pod \"global-pull-secret-syncer-d7fhl\" (UID: \"9ffc9cda-254d-4825-bb1a-1872331a8cc9\") " pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:18:53.314293 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:53.313983    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs\") pod \"network-metrics-daemon-fx9qn\" (UID: \"9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c\") " pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:18:53.314293 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:53.314090    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:53.314293 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:53.314143    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:18:53.314293 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:53.314185    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret podName:9ffc9cda-254d-4825-bb1a-1872331a8cc9 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:55.31416317 +0000 UTC m=+5.115705385 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret") pod "global-pull-secret-syncer-d7fhl" (UID: "9ffc9cda-254d-4825-bb1a-1872331a8cc9") : object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:53.314293 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:53.314209    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs podName:9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c nodeName:}" failed. No retries permitted until 2026-07-28 14:18:55.314197215 +0000 UTC m=+5.115739416 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs") pod "network-metrics-daemon-fx9qn" (UID: "9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:18:53.415351 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:53.415314    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlwv6\" (UniqueName: \"kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6\") pod \"network-check-target-lcn77\" (UID: \"dbe349f8-37a2-49ce-b92c-6100612b24b6\") " pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:18:53.415551 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:53.415489    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 14:18:53.415551 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:53.415513    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 14:18:53.415551 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:53.415526    2584 projected.go:196] Error preparing data for projected volume kube-api-access-vlwv6 for pod openshift-network-diagnostics/network-check-target-lcn77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:53.415709 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:53.415587    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6 podName:dbe349f8-37a2-49ce-b92c-6100612b24b6 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:55.415568595 +0000 UTC m=+5.217110805 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vlwv6" (UniqueName: "kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6") pod "network-check-target-lcn77" (UID: "dbe349f8-37a2-49ce-b92c-6100612b24b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:53.639807 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:53.639713    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 14:13:51 +0000 UTC" deadline="2028-05-01 22:04:55.07422643 +0000 UTC"
Jul 28 14:18:53.639807 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:53.639749    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15439h46m1.43448237s"
Jul 28 14:18:53.791565 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:53.790982    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:18:53.791565 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:53.791139    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx9qn" podUID="9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c"
Jul 28 14:18:54.344921 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:54.344820    2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 14:18:54.795609 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:54.790481    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:18:54.795609 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:54.790599    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7fhl" podUID="9ffc9cda-254d-4825-bb1a-1872331a8cc9"
Jul 28 14:18:54.795609 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:54.790994    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:18:54.795609 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:54.791134    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lcn77" podUID="dbe349f8-37a2-49ce-b92c-6100612b24b6"
Jul 28 14:18:55.333688 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:55.333408    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret\") pod \"global-pull-secret-syncer-d7fhl\" (UID: \"9ffc9cda-254d-4825-bb1a-1872331a8cc9\") " pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:18:55.333688 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:55.333469    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs\") pod \"network-metrics-daemon-fx9qn\" (UID: \"9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c\") " pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:18:55.333688 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:55.333558    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:55.333688 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:55.333582    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:18:55.333688 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:55.333633    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret podName:9ffc9cda-254d-4825-bb1a-1872331a8cc9 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:59.333613984 +0000 UTC m=+9.135156193 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret") pod "global-pull-secret-syncer-d7fhl" (UID: "9ffc9cda-254d-4825-bb1a-1872331a8cc9") : object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:55.333688 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:55.333654    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs podName:9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c nodeName:}" failed. No retries permitted until 2026-07-28 14:18:59.333642969 +0000 UTC m=+9.135185169 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs") pod "network-metrics-daemon-fx9qn" (UID: "9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:18:55.434039 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:55.433998    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlwv6\" (UniqueName: \"kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6\") pod \"network-check-target-lcn77\" (UID: \"dbe349f8-37a2-49ce-b92c-6100612b24b6\") " pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:18:55.434302 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:55.434230    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 14:18:55.434302 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:55.434254    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 14:18:55.434302 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:55.434267    2584 projected.go:196] Error preparing data for projected volume kube-api-access-vlwv6 for pod openshift-network-diagnostics/network-check-target-lcn77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:55.434502 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:55.434341    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6 podName:dbe349f8-37a2-49ce-b92c-6100612b24b6 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:59.434311834 +0000 UTC m=+9.235854040 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vlwv6" (UniqueName: "kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6") pod "network-check-target-lcn77" (UID: "dbe349f8-37a2-49ce-b92c-6100612b24b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:55.791178 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:55.790849    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:18:55.791178 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:55.790999    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx9qn" podUID="9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c"
Jul 28 14:18:56.795385 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:56.794833    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:18:56.795385 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:56.794844    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:18:56.795385 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:56.794965    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7fhl" podUID="9ffc9cda-254d-4825-bb1a-1872331a8cc9"
Jul 28 14:18:56.795385 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:56.795112    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lcn77" podUID="dbe349f8-37a2-49ce-b92c-6100612b24b6"
Jul 28 14:18:57.790620 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:57.790589    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:18:57.790802 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:57.790738    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx9qn" podUID="9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c"
Jul 28 14:18:58.790275 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:58.790195    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:18:58.790275 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:58.790242    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:18:58.790781 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:58.790340    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lcn77" podUID="dbe349f8-37a2-49ce-b92c-6100612b24b6"
Jul 28 14:18:58.790781 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:58.790452    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7fhl" podUID="9ffc9cda-254d-4825-bb1a-1872331a8cc9"
Jul 28 14:18:59.367966 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:59.367929    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret\") pod \"global-pull-secret-syncer-d7fhl\" (UID: \"9ffc9cda-254d-4825-bb1a-1872331a8cc9\") " pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:18:59.368154 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:59.367982    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs\") pod \"network-metrics-daemon-fx9qn\" (UID: \"9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c\") " pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:18:59.368154 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:59.368110    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:18:59.368154 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:59.368115    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:59.368384 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:59.368178    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs podName:9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c nodeName:}" failed. No retries permitted until 2026-07-28 14:19:07.368159335 +0000 UTC m=+17.169701531 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs") pod "network-metrics-daemon-fx9qn" (UID: "9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:18:59.368384 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:59.368197    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret podName:9ffc9cda-254d-4825-bb1a-1872331a8cc9 nodeName:}" failed. No retries permitted until 2026-07-28 14:19:07.368188112 +0000 UTC m=+17.169730306 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret") pod "global-pull-secret-syncer-d7fhl" (UID: "9ffc9cda-254d-4825-bb1a-1872331a8cc9") : object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:59.468681 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:59.468581    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlwv6\" (UniqueName: \"kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6\") pod \"network-check-target-lcn77\" (UID: \"dbe349f8-37a2-49ce-b92c-6100612b24b6\") " pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:18:59.468865 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:59.468773    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 14:18:59.468865 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:59.468801    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 14:18:59.468865 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:59.468817    2584 projected.go:196] Error preparing data for projected volume kube-api-access-vlwv6 for pod openshift-network-diagnostics/network-check-target-lcn77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:59.469003 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:59.468880    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6 podName:dbe349f8-37a2-49ce-b92c-6100612b24b6 nodeName:}" failed. No retries permitted until 2026-07-28 14:19:07.468860906 +0000 UTC m=+17.270403121 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vlwv6" (UniqueName: "kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6") pod "network-check-target-lcn77" (UID: "dbe349f8-37a2-49ce-b92c-6100612b24b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:59.790670 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:18:59.790586    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:18:59.791122 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:18:59.790743    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx9qn" podUID="9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c"
Jul 28 14:19:00.791494 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:00.791463    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:00.791943 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:00.791572    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7fhl" podUID="9ffc9cda-254d-4825-bb1a-1872331a8cc9"
Jul 28 14:19:00.791943 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:00.791667    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:00.791943 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:00.791777    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lcn77" podUID="dbe349f8-37a2-49ce-b92c-6100612b24b6"
Jul 28 14:19:01.791011 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:01.790969    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:19:01.791220 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:01.791123    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx9qn" podUID="9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c"
Jul 28 14:19:02.790061 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:02.790024    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:02.790465 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:02.790164    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lcn77" podUID="dbe349f8-37a2-49ce-b92c-6100612b24b6"
Jul 28 14:19:02.790465 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:02.790216    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:02.790465 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:02.790312    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7fhl" podUID="9ffc9cda-254d-4825-bb1a-1872331a8cc9"
Jul 28 14:19:03.790956 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:03.790919    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:19:03.791404 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:03.791106    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx9qn" podUID="9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c"
Jul 28 14:19:04.790194 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:04.790157    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:04.790353 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:04.790157    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:04.790353 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:04.790311    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7fhl" podUID="9ffc9cda-254d-4825-bb1a-1872331a8cc9"
Jul 28 14:19:04.790482 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:04.790361    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lcn77" podUID="dbe349f8-37a2-49ce-b92c-6100612b24b6"
Jul 28 14:19:05.790789 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:05.790755    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:19:05.791245 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:05.790871    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx9qn" podUID="9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c"
Jul 28 14:19:06.790992 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:06.790959    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:06.791450 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:06.791060    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lcn77" podUID="dbe349f8-37a2-49ce-b92c-6100612b24b6"
Jul 28 14:19:06.791450 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:06.791084    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:06.791450 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:06.791205    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7fhl" podUID="9ffc9cda-254d-4825-bb1a-1872331a8cc9"
Jul 28 14:19:07.427798 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:07.427756    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret\") pod \"global-pull-secret-syncer-d7fhl\" (UID: \"9ffc9cda-254d-4825-bb1a-1872331a8cc9\") " pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:07.428084 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:07.427807    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs\") pod \"network-metrics-daemon-fx9qn\" (UID: \"9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c\") " pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:19:07.428084 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:07.427923    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:19:07.428084 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:07.427991    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs podName:9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c nodeName:}" failed. No retries permitted until 2026-07-28 14:19:23.427969983 +0000 UTC m=+33.229512207 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs") pod "network-metrics-daemon-fx9qn" (UID: "9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:19:07.428084 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:07.427923    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 14:19:07.428325 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:07.428111    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret podName:9ffc9cda-254d-4825-bb1a-1872331a8cc9 nodeName:}" failed. No retries permitted until 2026-07-28 14:19:23.42807102 +0000 UTC m=+33.229613253 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret") pod "global-pull-secret-syncer-d7fhl" (UID: "9ffc9cda-254d-4825-bb1a-1872331a8cc9") : object "kube-system"/"original-pull-secret" not registered
Jul 28 14:19:07.528575 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:07.528532    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlwv6\" (UniqueName: \"kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6\") pod \"network-check-target-lcn77\" (UID: \"dbe349f8-37a2-49ce-b92c-6100612b24b6\") " pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:07.528748 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:07.528712    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 14:19:07.528748 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:07.528736    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 14:19:07.528748 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:07.528746    2584 projected.go:196] Error preparing data for projected volume kube-api-access-vlwv6 for pod openshift-network-diagnostics/network-check-target-lcn77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:19:07.528851 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:07.528797    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6 podName:dbe349f8-37a2-49ce-b92c-6100612b24b6 nodeName:}" failed. No retries permitted until 2026-07-28 14:19:23.528784075 +0000 UTC m=+33.330326271 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vlwv6" (UniqueName: "kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6") pod "network-check-target-lcn77" (UID: "dbe349f8-37a2-49ce-b92c-6100612b24b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:19:07.790269 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:07.790200    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:19:07.790409 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:07.790322    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx9qn" podUID="9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c"
Jul 28 14:19:08.790562 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:08.790523    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:08.791041 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:08.790657    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lcn77" podUID="dbe349f8-37a2-49ce-b92c-6100612b24b6"
Jul 28 14:19:08.791041 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:08.790710    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:08.791041 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:08.790818    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7fhl" podUID="9ffc9cda-254d-4825-bb1a-1872331a8cc9"
Jul 28 14:19:09.790541 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:09.790504    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:19:09.790744 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:09.790640    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx9qn" podUID="9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c"
Jul 28 14:19:10.791255 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:10.791219    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:10.791708 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:10.791335    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lcn77" podUID="dbe349f8-37a2-49ce-b92c-6100612b24b6"
Jul 28 14:19:10.791708 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:10.791389    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:10.791708 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:10.791511    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7fhl" podUID="9ffc9cda-254d-4825-bb1a-1872331a8cc9"
Jul 28 14:19:11.790893 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:11.790857    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:19:11.791062 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:11.791007    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx9qn" podUID="9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c"
Jul 28 14:19:12.790604 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:12.790274    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:12.791446 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:12.790686    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lcn77" podUID="dbe349f8-37a2-49ce-b92c-6100612b24b6"
Jul 28 14:19:12.791446 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:12.790342    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:12.791446 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:12.790762    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7fhl" podUID="9ffc9cda-254d-4825-bb1a-1872331a8cc9"
Jul 28 14:19:12.974494 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:12.974462    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rsxdj" event={"ID":"e1374453-20fd-4a88-bef4-82e998969586","Type":"ContainerStarted","Data":"90c167b7ba5a3f3bad7e16f3a37f9c45c39ea04a96361d6446f6cdd8f724e319"}
Jul 28 14:19:12.976139 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:12.976110    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k66dk" event={"ID":"d3c1ce2b-ee28-4657-81d3-a2f28736fab4","Type":"ContainerStarted","Data":"4b9aaa8b5648a1a6c75b9b139649c067b764786753cee2494eadb648ed326c9b"}
Jul 28 14:19:12.977453 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:12.977415    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h2ddz" event={"ID":"d943e044-62ab-4f2c-a9a6-10fc30a01aca","Type":"ContainerStarted","Data":"2eb2934d744269f1361a27b4175543124883a530bce22830552ac335d38e1951"}
Jul 28 14:19:12.978933 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:12.978909    2584 generic.go:356] "Generic (PLEG): container finished" podID="c2dfdeb6-5821-484f-8548-5d305d57a790" containerID="0c36f91574ebe186245f40e61b724dc7114f4e0c6da2d25b2039d23de34a2079" exitCode=0
Jul 28 14:19:12.979031 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:12.978979    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p789p" event={"ID":"c2dfdeb6-5821-484f-8548-5d305d57a790","Type":"ContainerDied","Data":"0c36f91574ebe186245f40e61b724dc7114f4e0c6da2d25b2039d23de34a2079"}
Jul 28 14:19:12.980586 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:12.980540    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-df54l" event={"ID":"26358014-f39f-48a3-94df-fcc8c7ba60dc","Type":"ContainerStarted","Data":"f0948696d202f8106a4a6ae70d4ab196df9e26ed1b04ea3b0ef06a08801e045e"}
Jul 28 14:19:12.982175 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:12.982152    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn" event={"ID":"c54a401c-f28b-49df-9323-26d3ee747317","Type":"ContainerStarted","Data":"d890167694c85e58bc4bff0be9a4391c776e3e51489070d7b926b22ea11eb575"}
Jul 28 14:19:12.983641 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:12.983611    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal" event={"ID":"f675f063811c81df70fa063df9f70b5b","Type":"ContainerStarted","Data":"ddeca0cf17b9a53fd56ff7bcdf540af304d6aec46bd26e73938e156249c22956"}
Jul 28 14:19:12.985121 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:12.985079    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jldlr" event={"ID":"1c2b854d-bbb4-4b63-b880-f71b7df99fdd","Type":"ContainerStarted","Data":"d2643a79f6656ae90b11ca5b8878948168f729e2658e5b0344a226edea39afe3"}
Jul 28 14:19:12.988067 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:12.988048    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc" event={"ID":"8c78f0b5-33e8-40ec-994e-9c7d9448ac74","Type":"ContainerStarted","Data":"944936f3c04c95a7ab1dbf4f1d7a0a59f4d83a57afca2c734b771b2398edbf49"}
Jul 28 14:19:12.988179 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:12.988133    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc" event={"ID":"8c78f0b5-33e8-40ec-994e-9c7d9448ac74","Type":"ContainerStarted","Data":"a32e2837b76a04d17def20a578a296aec3b43b9701845b09c4a19ced959ff490"}
Jul 28 14:19:12.988179 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:12.988177    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc" event={"ID":"8c78f0b5-33e8-40ec-994e-9c7d9448ac74","Type":"ContainerStarted","Data":"7d6626168caa380a791bedfa6546c7638d201239878a27fa8f60f752317d63ca"}
Jul 28 14:19:12.988291 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:12.988222    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc" event={"ID":"8c78f0b5-33e8-40ec-994e-9c7d9448ac74","Type":"ContainerStarted","Data":"4e409a56bec08edd29d5e7d97028d22fb7819191680cd5ab4e66f95755c904b9"}
Jul 28 14:19:12.988291 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:12.988252    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc" event={"ID":"8c78f0b5-33e8-40ec-994e-9c7d9448ac74","Type":"ContainerStarted","Data":"7eb91588b799129a47c16b9fac56d8ccc8b5bfc9cad743bdee9950957e482aa0"}
Jul 28 14:19:12.988389 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:12.988296    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc" event={"ID":"8c78f0b5-33e8-40ec-994e-9c7d9448ac74","Type":"ContainerStarted","Data":"748f8b2f4265cb01fa5d2600af26a3f595309eb9d9b11901059b0674ef84e92b"}
Jul 28 14:19:12.989634 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:12.989611    2584 generic.go:356] "Generic (PLEG): container finished" podID="7e6933567c76424165705d8ba7f1f8be" containerID="b3348bb80a48d6a836599688159e5a799840c92046d8620dde30378c3245afb6" exitCode=0
Jul 28 14:19:12.989707 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:12.989641    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal" event={"ID":"7e6933567c76424165705d8ba7f1f8be","Type":"ContainerDied","Data":"b3348bb80a48d6a836599688159e5a799840c92046d8620dde30378c3245afb6"}
Jul 28 14:19:13.000254 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:13.000212    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rsxdj" podStartSLOduration=1.8880748920000001 podStartE2EDuration="22.000196898s" totalImagesPullingTime="20.112122006s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:12.998307318 +0000 UTC m=+22.799849532" watchObservedRunningTime="2026-07-28 14:19:13.000196898 +0000 UTC m=+22.801739115"
Jul 28 14:19:13.020915 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:13.019219    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-df54l" podStartSLOduration=1.8678441609999998 podStartE2EDuration="22.019195625s" totalImagesPullingTime="20.151351464s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:13.01791668 +0000 UTC m=+22.819458890" watchObservedRunningTime="2026-07-28 14:19:13.019195625 +0000 UTC m=+22.820737841"
Jul 28 14:19:13.069103 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:13.069049    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-h2ddz" podStartSLOduration=1.96762547 podStartE2EDuration="22.069031925s" totalImagesPullingTime="20.101406455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:13.068027472 +0000 UTC m=+22.869569689" watchObservedRunningTime="2026-07-28 14:19:13.069031925 +0000 UTC m=+22.870574139"
Jul 28 14:19:13.105359 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:13.105263    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-k66dk" podStartSLOduration=2.004024555 podStartE2EDuration="22.105246571s" totalImagesPullingTime="20.101222016s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:13.104864658 +0000 UTC m=+22.906406873" watchObservedRunningTime="2026-07-28 14:19:13.105246571 +0000 UTC m=+22.906788787"
Jul 28 14:19:13.165151 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:13.165079    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jldlr" podStartSLOduration=2.084368869 podStartE2EDuration="22.165062459s" totalImagesPullingTime="20.08069359s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:13.130165186 +0000 UTC m=+22.931707400" watchObservedRunningTime="2026-07-28 14:19:13.165062459 +0000 UTC m=+22.966604692"
Jul 28 14:19:13.790859 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:13.790774    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:19:13.791390 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:13.790906    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx9qn" podUID="9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c"
Jul 28 14:19:13.994197 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:13.994157    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j746c" event={"ID":"b232405a-e46c-4944-ac1a-954bfef6190e","Type":"ContainerStarted","Data":"ce9aaf93a96e5f19fba2e683e7d7c96f0f8ff021b8bbe1f06e8ccca14a124e56"}
Jul 28 14:19:14.011304 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:14.011246    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal" podStartSLOduration=23.011224781 podStartE2EDuration="23.011224781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:13.164698195 +0000 UTC m=+22.966240409" watchObservedRunningTime="2026-07-28 14:19:14.011224781 +0000 UTC m=+23.812766997"
Jul 28 14:19:14.027112 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:14.027048    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-j746c" podStartSLOduration=2.8708036999999997 podStartE2EDuration="23.027031344s" totalImagesPullingTime="20.156227644s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:14.026650462 +0000 UTC m=+23.828192677" watchObservedRunningTime="2026-07-28 14:19:14.027031344 +0000 UTC m=+23.828573558"
Jul 28 14:19:14.790698 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:14.790662    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:14.790698 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:14.790692    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:14.791283 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:14.790805    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lcn77" podUID="dbe349f8-37a2-49ce-b92c-6100612b24b6"
Jul 28 14:19:14.791283 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:14.790924    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7fhl" podUID="9ffc9cda-254d-4825-bb1a-1872331a8cc9"
Jul 28 14:19:14.998468 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:14.998438    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal" event={"ID":"7e6933567c76424165705d8ba7f1f8be","Type":"ContainerStarted","Data":"8ac746bb767a89d9ccc7c048b6cd7e729132bfe666091e13e6f2d2ca6e5a0628"}
Jul 28 14:19:15.020565 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:15.020513    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal" podStartSLOduration=24.020498449 podStartE2EDuration="24.020498449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:15.020366167 +0000 UTC m=+24.821908382" watchObservedRunningTime="2026-07-28 14:19:15.020498449 +0000 UTC m=+24.822040663"
Jul 28 14:19:15.790298 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:15.790210    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:19:15.790456 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:15.790352    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx9qn" podUID="9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c"
Jul 28 14:19:16.004730 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:16.004686    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc" event={"ID":"8c78f0b5-33e8-40ec-994e-9c7d9448ac74","Type":"ContainerStarted","Data":"182ad643cbf37f1137b7feae261c78a4128744fa5006fc87124dfbda000a4c6d"}
Jul 28 14:19:16.794825 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:16.794796    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:16.795029 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:16.794922    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7fhl" podUID="9ffc9cda-254d-4825-bb1a-1872331a8cc9"
Jul 28 14:19:16.795029 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:16.794961    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:16.795169 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:16.795069    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lcn77" podUID="dbe349f8-37a2-49ce-b92c-6100612b24b6"
Jul 28 14:19:16.919601 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:16.919557    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-h2ddz"
Jul 28 14:19:16.920237 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:16.920215    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-h2ddz"
Jul 28 14:19:17.007233 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:17.007200    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-h2ddz"
Jul 28 14:19:17.007823 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:17.007787    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-h2ddz"
Jul 28 14:19:17.214478 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:17.214319    2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 14:19:17.673361 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:17.673205    2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T14:19:17.214469636Z","UUID":"e9eb2d4f-7d94-4b14-ad33-8696aacf9032","Handler":null,"Name":"","Endpoint":""}
Jul 28 14:19:17.675083 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:17.675063    2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 14:19:17.675227 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:17.675110    2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 14:19:17.790976 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:17.790945    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:19:17.791145 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:17.791069    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx9qn" podUID="9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c"
Jul 28 14:19:18.012239 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:18.012152    2584 generic.go:356] "Generic (PLEG): container finished" podID="c2dfdeb6-5821-484f-8548-5d305d57a790" containerID="2f748aec50f5728a25587d65d9d52bb7cc24e814d8bcceba9b9e63ce0a4b27b6" exitCode=0
Jul 28 14:19:18.013075 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:18.012228    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p789p" event={"ID":"c2dfdeb6-5821-484f-8548-5d305d57a790","Type":"ContainerDied","Data":"2f748aec50f5728a25587d65d9d52bb7cc24e814d8bcceba9b9e63ce0a4b27b6"}
Jul 28 14:19:18.014276 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:18.014254    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn" event={"ID":"c54a401c-f28b-49df-9323-26d3ee747317","Type":"ContainerStarted","Data":"fe1b03e40e9301a905102bf12471f7752e9839b6e34f1706bd0741fd73e27dea"}
Jul 28 14:19:18.018263 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:18.018233    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc" event={"ID":"8c78f0b5-33e8-40ec-994e-9c7d9448ac74","Type":"ContainerStarted","Data":"d1a383e4b955c0533dbe6060586e71b7dd99693368ec47b367be544e6ad9946b"}
Jul 28 14:19:18.071990 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:18.071937    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc" podStartSLOduration=6.864106232 podStartE2EDuration="27.071920405s" totalImagesPullingTime="20.207814173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:18.071687619 +0000 UTC m=+27.873229834" watchObservedRunningTime="2026-07-28 14:19:18.071920405 +0000 UTC m=+27.873462620"
Jul 28 14:19:18.791007 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:18.790741    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:18.791194 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:18.790825    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:18.791194 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:18.791048    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lcn77" podUID="dbe349f8-37a2-49ce-b92c-6100612b24b6"
Jul 28 14:19:18.791194 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:18.791135    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7fhl" podUID="9ffc9cda-254d-4825-bb1a-1872331a8cc9"
Jul 28 14:19:19.022106 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:19.022055    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn" event={"ID":"c54a401c-f28b-49df-9323-26d3ee747317","Type":"ContainerStarted","Data":"63e89e34d780acb10bc867dc5c2161cdaa701bc13400099a3ff6cdf2adb21c2d"}
Jul 28 14:19:19.029485 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:19.029462    2584 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 14:19:19.029887 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:19.029856    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:19:19.030002 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:19.029942    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:19:19.047425 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:19.047362    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:19:19.047548 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:19.047516    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:19:19.077925 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:19.077876    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7fhwn" podStartSLOduration=2.130372172 podStartE2EDuration="28.077858234s" totalImagesPullingTime="25.947486062s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:19.04811593 +0000 UTC m=+28.849658146" watchObservedRunningTime="2026-07-28 14:19:19.077858234 +0000 UTC m=+28.879400449"
Jul 28 14:19:19.720939 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:19.720907    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d7fhl"]
Jul 28 14:19:19.721135 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:19.721039    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:19.721179 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:19.721137    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7fhl" podUID="9ffc9cda-254d-4825-bb1a-1872331a8cc9"
Jul 28 14:19:19.726141 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:19.726116    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lcn77"]
Jul 28 14:19:19.726279 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:19.726232    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:19.726344 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:19.726327    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lcn77" podUID="dbe349f8-37a2-49ce-b92c-6100612b24b6"
Jul 28 14:19:19.726975 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:19.726953    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fx9qn"]
Jul 28 14:19:19.727083 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:19.727069    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:19:19.727182 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:19.727164    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx9qn" podUID="9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c"
Jul 28 14:19:20.026040 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:20.026008    2584 generic.go:356] "Generic (PLEG): container finished" podID="c2dfdeb6-5821-484f-8548-5d305d57a790" containerID="2ad67293fff7e4ea5ce3a949fd575a682a1254c5a5429e488a3acc0c9fc41d06" exitCode=0
Jul 28 14:19:20.026485 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:20.026108    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p789p" event={"ID":"c2dfdeb6-5821-484f-8548-5d305d57a790","Type":"ContainerDied","Data":"2ad67293fff7e4ea5ce3a949fd575a682a1254c5a5429e488a3acc0c9fc41d06"}
Jul 28 14:19:20.026485 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:20.026171    2584 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 14:19:21.027604 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:21.027569    2584 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 14:19:21.790852 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:21.790813    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:21.791018 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:21.790813    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:19:21.791018 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:21.790927    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7fhl" podUID="9ffc9cda-254d-4825-bb1a-1872331a8cc9"
Jul 28 14:19:21.791102 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:21.790823    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:21.791102 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:21.791059    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx9qn" podUID="9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c"
Jul 28 14:19:21.791173 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:21.791152    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lcn77" podUID="dbe349f8-37a2-49ce-b92c-6100612b24b6"
Jul 28 14:19:21.918616 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:21.918573    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:19:22.032699 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:22.032660    2584 generic.go:356] "Generic (PLEG): container finished" podID="c2dfdeb6-5821-484f-8548-5d305d57a790" containerID="a494af6bdd33c5dc78430abfb9c29c162f524013367df628f74959a8f9f62edf" exitCode=0
Jul 28 14:19:22.033160 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:22.032742    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p789p" event={"ID":"c2dfdeb6-5821-484f-8548-5d305d57a790","Type":"ContainerDied","Data":"a494af6bdd33c5dc78430abfb9c29c162f524013367df628f74959a8f9f62edf"}
Jul 28 14:19:22.045663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:22.045623    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc" podUID="8c78f0b5-33e8-40ec-994e-9c7d9448ac74" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 28 14:19:23.454260 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:23.454225    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret\") pod \"global-pull-secret-syncer-d7fhl\" (UID: \"9ffc9cda-254d-4825-bb1a-1872331a8cc9\") " pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:23.454260 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:23.454268    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs\") pod \"network-metrics-daemon-fx9qn\" (UID: \"9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c\") " pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:19:23.454760 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:23.454388    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:19:23.454760 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:23.454450    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs podName:9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c nodeName:}" failed. No retries permitted until 2026-07-28 14:19:55.454435708 +0000 UTC m=+65.255977901 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs") pod "network-metrics-daemon-fx9qn" (UID: "9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:19:23.454760 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:23.454388    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 14:19:23.454760 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:23.454527    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret podName:9ffc9cda-254d-4825-bb1a-1872331a8cc9 nodeName:}" failed. No retries permitted until 2026-07-28 14:19:55.454514574 +0000 UTC m=+65.256056785 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret") pod "global-pull-secret-syncer-d7fhl" (UID: "9ffc9cda-254d-4825-bb1a-1872331a8cc9") : object "kube-system"/"original-pull-secret" not registered
Jul 28 14:19:23.555167 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:23.555127    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlwv6\" (UniqueName: \"kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6\") pod \"network-check-target-lcn77\" (UID: \"dbe349f8-37a2-49ce-b92c-6100612b24b6\") " pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:23.555371 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:23.555317    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 14:19:23.555371 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:23.555338    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 14:19:23.555371 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:23.555348    2584 projected.go:196] Error preparing data for projected volume kube-api-access-vlwv6 for pod openshift-network-diagnostics/network-check-target-lcn77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:19:23.555529 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:23.555402    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6 podName:dbe349f8-37a2-49ce-b92c-6100612b24b6 nodeName:}" failed. No retries permitted until 2026-07-28 14:19:55.555389238 +0000 UTC m=+65.356931431 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vlwv6" (UniqueName: "kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6") pod "network-check-target-lcn77" (UID: "dbe349f8-37a2-49ce-b92c-6100612b24b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:19:23.791074 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:23.790996    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:23.791074 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:23.791026    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:23.791074 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:23.791026    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:19:23.791301 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:23.791165    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7fhl" podUID="9ffc9cda-254d-4825-bb1a-1872331a8cc9"
Jul 28 14:19:23.791301 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:23.791267    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lcn77" podUID="dbe349f8-37a2-49ce-b92c-6100612b24b6"
Jul 28 14:19:23.791385 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:23.791328    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx9qn" podUID="9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c"
Jul 28 14:19:25.791176 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:25.790961    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:25.791591 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:25.790961    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:19:25.791591 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:25.790977    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:25.791591 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:25.791270    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7fhl" podUID="9ffc9cda-254d-4825-bb1a-1872331a8cc9"
Jul 28 14:19:25.791591 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:25.791374    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lcn77" podUID="dbe349f8-37a2-49ce-b92c-6100612b24b6"
Jul 28 14:19:25.791591 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:25.791465    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx9qn" podUID="9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c"
Jul 28 14:19:26.008310 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.008233    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeReady"
Jul 28 14:19:26.008470 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.008382    2584 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 14:19:26.060356 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.060316    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hkt4c"]
Jul 28 14:19:26.062730 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.062701    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:26.063412 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.063391    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-m2qz5"]
Jul 28 14:19:26.066456 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.065495    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m2qz5"
Jul 28 14:19:26.066456 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.065663    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 14:19:26.066456 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.065967    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 14:19:26.066456 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.066165    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6f28j\""
Jul 28 14:19:26.068474 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.068446    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 14:19:26.068590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.068510    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 14:19:26.068590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.068518    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 14:19:26.068810 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.068786    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zfw4b\""
Jul 28 14:19:26.080238 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.080147    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hkt4c"]
Jul 28 14:19:26.080987 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.080960    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m2qz5"]
Jul 28 14:19:26.175744 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.175705    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls\") pod \"dns-default-hkt4c\" (UID: \"8dc38f88-df97-4d9f-8718-306934c61e2b\") " pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:26.175894 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.175763    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkqht\" (UniqueName: \"kubernetes.io/projected/8dc38f88-df97-4d9f-8718-306934c61e2b-kube-api-access-tkqht\") pod \"dns-default-hkt4c\" (UID: \"8dc38f88-df97-4d9f-8718-306934c61e2b\") " pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:26.175894 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.175784    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m2h9\" (UniqueName: \"kubernetes.io/projected/e78ad162-6c3b-41de-ad16-9f39ce98f98e-kube-api-access-8m2h9\") pod \"ingress-canary-m2qz5\" (UID: \"e78ad162-6c3b-41de-ad16-9f39ce98f98e\") " pod="openshift-ingress-canary/ingress-canary-m2qz5"
Jul 28 14:19:26.175894 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.175810    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8dc38f88-df97-4d9f-8718-306934c61e2b-config-volume\") pod \"dns-default-hkt4c\" (UID: \"8dc38f88-df97-4d9f-8718-306934c61e2b\") " pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:26.175894 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.175830    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e78ad162-6c3b-41de-ad16-9f39ce98f98e-cert\") pod \"ingress-canary-m2qz5\" (UID: \"e78ad162-6c3b-41de-ad16-9f39ce98f98e\") " pod="openshift-ingress-canary/ingress-canary-m2qz5"
Jul 28 14:19:26.176071 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.175894    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dc38f88-df97-4d9f-8718-306934c61e2b-tmp-dir\") pod \"dns-default-hkt4c\" (UID: \"8dc38f88-df97-4d9f-8718-306934c61e2b\") " pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:26.277270 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.277194    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls\") pod \"dns-default-hkt4c\" (UID: \"8dc38f88-df97-4d9f-8718-306934c61e2b\") " pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:26.277270 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.277255    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkqht\" (UniqueName: \"kubernetes.io/projected/8dc38f88-df97-4d9f-8718-306934c61e2b-kube-api-access-tkqht\") pod \"dns-default-hkt4c\" (UID: \"8dc38f88-df97-4d9f-8718-306934c61e2b\") " pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:26.277270 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.277273    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m2h9\" (UniqueName: \"kubernetes.io/projected/e78ad162-6c3b-41de-ad16-9f39ce98f98e-kube-api-access-8m2h9\") pod \"ingress-canary-m2qz5\" (UID: \"e78ad162-6c3b-41de-ad16-9f39ce98f98e\") " pod="openshift-ingress-canary/ingress-canary-m2qz5"
Jul 28 14:19:26.277471 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.277306    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8dc38f88-df97-4d9f-8718-306934c61e2b-config-volume\") pod \"dns-default-hkt4c\" (UID: \"8dc38f88-df97-4d9f-8718-306934c61e2b\") " pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:26.277471 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.277328    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e78ad162-6c3b-41de-ad16-9f39ce98f98e-cert\") pod \"ingress-canary-m2qz5\" (UID: \"e78ad162-6c3b-41de-ad16-9f39ce98f98e\") " pod="openshift-ingress-canary/ingress-canary-m2qz5"
Jul 28 14:19:26.277471 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.277344    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dc38f88-df97-4d9f-8718-306934c61e2b-tmp-dir\") pod \"dns-default-hkt4c\" (UID: \"8dc38f88-df97-4d9f-8718-306934c61e2b\") " pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:26.277471 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:26.277351    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 14:19:26.277471 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:26.277456    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls podName:8dc38f88-df97-4d9f-8718-306934c61e2b nodeName:}" failed. No retries permitted until 2026-07-28 14:19:26.777407239 +0000 UTC m=+36.578949455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls") pod "dns-default-hkt4c" (UID: "8dc38f88-df97-4d9f-8718-306934c61e2b") : secret "dns-default-metrics-tls" not found
Jul 28 14:19:26.277712 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.277637    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dc38f88-df97-4d9f-8718-306934c61e2b-tmp-dir\") pod \"dns-default-hkt4c\" (UID: \"8dc38f88-df97-4d9f-8718-306934c61e2b\") " pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:26.277854 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.277834    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8dc38f88-df97-4d9f-8718-306934c61e2b-config-volume\") pod \"dns-default-hkt4c\" (UID: \"8dc38f88-df97-4d9f-8718-306934c61e2b\") " pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:26.281661 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.281635    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e78ad162-6c3b-41de-ad16-9f39ce98f98e-cert\") pod \"ingress-canary-m2qz5\" (UID: \"e78ad162-6c3b-41de-ad16-9f39ce98f98e\") " pod="openshift-ingress-canary/ingress-canary-m2qz5"
Jul 28 14:19:26.289445 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.289406    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkqht\" (UniqueName: \"kubernetes.io/projected/8dc38f88-df97-4d9f-8718-306934c61e2b-kube-api-access-tkqht\") pod \"dns-default-hkt4c\" (UID: \"8dc38f88-df97-4d9f-8718-306934c61e2b\") " pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:26.289641 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.289609    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m2h9\" (UniqueName: \"kubernetes.io/projected/e78ad162-6c3b-41de-ad16-9f39ce98f98e-kube-api-access-8m2h9\") pod \"ingress-canary-m2qz5\" (UID: \"e78ad162-6c3b-41de-ad16-9f39ce98f98e\") " pod="openshift-ingress-canary/ingress-canary-m2qz5"
Jul 28 14:19:26.383854 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.383804    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m2qz5"
Jul 28 14:19:26.518830 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.518799    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m2qz5"]
Jul 28 14:19:26.522623 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:19:26.522592    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode78ad162_6c3b_41de_ad16_9f39ce98f98e.slice/crio-0273dcb86b692e81c87825a3675b9d268511801b8b39c2e2d3fe2cfc46675144 WatchSource:0}: Error finding container 0273dcb86b692e81c87825a3675b9d268511801b8b39c2e2d3fe2cfc46675144: Status 404 returned error can't find the container with id 0273dcb86b692e81c87825a3675b9d268511801b8b39c2e2d3fe2cfc46675144
Jul 28 14:19:26.650026 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.649948    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cbx4k"]
Jul 28 14:19:26.652502 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.652477    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:26.659511 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.659481    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 14:19:26.659798 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.659543    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 14:19:26.659868 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.659828    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 14:19:26.660078 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.660058    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kr6cm\""
Jul 28 14:19:26.663555 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.663531    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 14:19:26.687693 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.687659    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cbx4k"]
Jul 28 14:19:26.780660 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.780616    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-crio-socket\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:26.780805 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.780675    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls\") pod \"dns-default-hkt4c\" (UID: \"8dc38f88-df97-4d9f-8718-306934c61e2b\") " pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:26.780805 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.780728    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-data-volume\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:26.780805 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.780743    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:26.780805 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.780758    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:26.780950 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.780804    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsgzx\" (UniqueName: \"kubernetes.io/projected/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-api-access-lsgzx\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:26.780950 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:26.780838    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 14:19:26.780950 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:26.780913    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls podName:8dc38f88-df97-4d9f-8718-306934c61e2b nodeName:}" failed. No retries permitted until 2026-07-28 14:19:27.780891034 +0000 UTC m=+37.582433228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls") pod "dns-default-hkt4c" (UID: "8dc38f88-df97-4d9f-8718-306934c61e2b") : secret "dns-default-metrics-tls" not found
Jul 28 14:19:26.882112 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.882044    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-data-volume\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:26.882112 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.882115    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:26.882897 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.882140    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:26.882897 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.882161    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsgzx\" (UniqueName: \"kubernetes.io/projected/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-api-access-lsgzx\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:26.882897 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.882204    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-crio-socket\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:26.882897 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:26.882251    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 14:19:26.882897 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:26.882295    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:26.882897 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:26.882340    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-rbac-proxy-cm podName:d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d nodeName:}" failed. No retries permitted until 2026-07-28 14:19:27.382319533 +0000 UTC m=+37.183861740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cbx4k" (UID: "d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d") : configmap "kube-rbac-proxy" not found
Jul 28 14:19:26.882897 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:26.882359    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls podName:d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d nodeName:}" failed. No retries permitted until 2026-07-28 14:19:27.382350764 +0000 UTC m=+37.183892957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cbx4k" (UID: "d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d") : secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:26.882897 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.882435    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-crio-socket\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:26.882897 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.882435    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-data-volume\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:26.890989 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:26.890964    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsgzx\" (UniqueName: \"kubernetes.io/projected/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-api-access-lsgzx\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:27.049008 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:27.048898    2584 generic.go:356] "Generic (PLEG): container finished" podID="c2dfdeb6-5821-484f-8548-5d305d57a790" containerID="2399415b54a482cb6d1bd55c4c6251e1e0873d65ead3ca52310a970594f6486e" exitCode=0
Jul 28 14:19:27.049008 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:27.048992    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p789p" event={"ID":"c2dfdeb6-5821-484f-8548-5d305d57a790","Type":"ContainerDied","Data":"2399415b54a482cb6d1bd55c4c6251e1e0873d65ead3ca52310a970594f6486e"}
Jul 28 14:19:27.050627 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:27.050598    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m2qz5" event={"ID":"e78ad162-6c3b-41de-ad16-9f39ce98f98e","Type":"ContainerStarted","Data":"0273dcb86b692e81c87825a3675b9d268511801b8b39c2e2d3fe2cfc46675144"}
Jul 28 14:19:27.387716 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:27.387627    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:27.387716 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:27.387663    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:27.387944 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:27.387777    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 14:19:27.387944 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:27.387797    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:27.387944 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:27.387858    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-rbac-proxy-cm podName:d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d nodeName:}" failed. No retries permitted until 2026-07-28 14:19:28.387837987 +0000 UTC m=+38.189380197 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cbx4k" (UID: "d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d") : configmap "kube-rbac-proxy" not found
Jul 28 14:19:27.387944 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:27.387878    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls podName:d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d nodeName:}" failed. No retries permitted until 2026-07-28 14:19:28.387868084 +0000 UTC m=+38.189410282 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cbx4k" (UID: "d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d") : secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:27.790469 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:27.790379    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:27.790469 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:27.790402    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:19:27.790701 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:27.790379    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:27.790701 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:27.790581    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls\") pod \"dns-default-hkt4c\" (UID: \"8dc38f88-df97-4d9f-8718-306934c61e2b\") " pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:27.790862 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:27.790739    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 14:19:27.790862 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:27.790783    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls podName:8dc38f88-df97-4d9f-8718-306934c61e2b nodeName:}" failed. No retries permitted until 2026-07-28 14:19:29.790770053 +0000 UTC m=+39.592312246 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls") pod "dns-default-hkt4c" (UID: "8dc38f88-df97-4d9f-8718-306934c61e2b") : secret "dns-default-metrics-tls" not found
Jul 28 14:19:27.794065 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:27.794035    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 14:19:27.794265 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:27.794244    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 14:19:27.794523 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:27.794505    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4gd6j\""
Jul 28 14:19:27.795126 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:27.794690    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 14:19:27.795126 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:27.794888    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v4vsp\""
Jul 28 14:19:27.795291 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:27.795141    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 14:19:28.057577 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:28.057475    2584 generic.go:356] "Generic (PLEG): container finished" podID="c2dfdeb6-5821-484f-8548-5d305d57a790" containerID="44cc798041dd7fdcf2eef7bd61559c0918b6abdbaecdb5387570314b4b8555d7" exitCode=0
Jul 28 14:19:28.058004 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:28.057572    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p789p" event={"ID":"c2dfdeb6-5821-484f-8548-5d305d57a790","Type":"ContainerDied","Data":"44cc798041dd7fdcf2eef7bd61559c0918b6abdbaecdb5387570314b4b8555d7"}
Jul 28 14:19:28.398878 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:28.398784    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:28.398878 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:28.398829    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:28.399138 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:28.398920    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 14:19:28.399138 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:28.398947    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:28.399138 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:28.399000    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-rbac-proxy-cm podName:d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d nodeName:}" failed. No retries permitted until 2026-07-28 14:19:30.39897918 +0000 UTC m=+40.200521391 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cbx4k" (UID: "d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d") : configmap "kube-rbac-proxy" not found
Jul 28 14:19:28.399138 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:28.399018    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls podName:d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d nodeName:}" failed. No retries permitted until 2026-07-28 14:19:30.399010181 +0000 UTC m=+40.200552378 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cbx4k" (UID: "d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d") : secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:28.658565 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:28.658484    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k66dk_d3c1ce2b-ee28-4657-81d3-a2f28736fab4/dns-node-resolver/0.log"
Jul 28 14:19:29.066791 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:29.066704    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p789p" event={"ID":"c2dfdeb6-5821-484f-8548-5d305d57a790","Type":"ContainerStarted","Data":"430818d1673cd7e37c29854bd4c6b942f868b6a3d3d4944633c7f807440525aa"}
Jul 28 14:19:29.096602 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:29.096538    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-p789p" podStartSLOduration=7.408464739 podStartE2EDuration="38.096515776s" totalImagesPullingTime="30.688051037s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:29.095355093 +0000 UTC m=+38.896897307" watchObservedRunningTime="2026-07-28 14:19:29.096515776 +0000 UTC m=+38.898057992"
Jul 28 14:19:29.259025 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:29.258860    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jldlr_1c2b854d-bbb4-4b63-b880-f71b7df99fdd/node-ca/0.log"
Jul 28 14:19:29.811612 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:29.811573    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls\") pod \"dns-default-hkt4c\" (UID: \"8dc38f88-df97-4d9f-8718-306934c61e2b\") " pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:29.811800 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:29.811694    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 14:19:29.811800 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:29.811750    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls podName:8dc38f88-df97-4d9f-8718-306934c61e2b nodeName:}" failed. No retries permitted until 2026-07-28 14:19:33.811735312 +0000 UTC m=+43.613277505 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls") pod "dns-default-hkt4c" (UID: "8dc38f88-df97-4d9f-8718-306934c61e2b") : secret "dns-default-metrics-tls" not found
Jul 28 14:19:30.071022 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:30.070934    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m2qz5" event={"ID":"e78ad162-6c3b-41de-ad16-9f39ce98f98e","Type":"ContainerStarted","Data":"faa5ad1789d19b427b70892e3e66ebf69e538f3007add65b352c202d7bb8ca14"}
Jul 28 14:19:30.259051 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:30.259021    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m2qz5_e78ad162-6c3b-41de-ad16-9f39ce98f98e/serve-healthcheck-canary/0.log"
Jul 28 14:19:30.416961 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:30.416874    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:30.416961 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:30.416915    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:30.417155 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:30.416995    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 14:19:30.417155 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:30.417061    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-rbac-proxy-cm podName:d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d nodeName:}" failed. No retries permitted until 2026-07-28 14:19:34.417046297 +0000 UTC m=+44.218588505 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cbx4k" (UID: "d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d") : configmap "kube-rbac-proxy" not found
Jul 28 14:19:30.417155 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:30.417087    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:30.417260 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:30.417164    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls podName:d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d nodeName:}" failed. No retries permitted until 2026-07-28 14:19:34.417149039 +0000 UTC m=+44.218691249 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cbx4k" (UID: "d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d") : secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:32.357651 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.355113    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-m2qz5" podStartSLOduration=3.69238516 podStartE2EDuration="6.355072205s" totalImagesPullingTime="2.662687045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:19:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:30.091154158 +0000 UTC m=+39.892696374" watchObservedRunningTime="2026-07-28 14:19:32.355072205 +0000 UTC m=+42.156614422"
Jul 28 14:19:32.357651 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.356814    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85"]
Jul 28 14:19:32.386393 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.386363    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7vw6"]
Jul 28 14:19:32.386539 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.386494    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85"
Jul 28 14:19:32.389541 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.389519    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 28 14:19:32.389693 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.389672    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-sp85h\""
Jul 28 14:19:32.389693 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.389686    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 28 14:19:32.389805 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.389679    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 28 14:19:32.389805 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.389733    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 28 14:19:32.406452 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.406410    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7vw6"]
Jul 28 14:19:32.406452 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.406462    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85"]
Jul 28 14:19:32.406675 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.406545    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7vw6"
Jul 28 14:19:32.409804 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.409776    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 28 14:19:32.409942 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.409870    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 28 14:19:32.420037 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.420014    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-zdqbl\""
Jul 28 14:19:32.534687 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.534648    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e793c22a-52b4-4605-ab1b-93488f784510-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85\" (UID: \"e793c22a-52b4-4605-ab1b-93488f784510\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85"
Jul 28 14:19:32.534687 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.534686    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvsdd\" (UniqueName: \"kubernetes.io/projected/e793c22a-52b4-4605-ab1b-93488f784510-kube-api-access-jvsdd\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85\" (UID: \"e793c22a-52b4-4605-ab1b-93488f784510\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85"
Jul 28 14:19:32.534874 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.534725    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e793c22a-52b4-4605-ab1b-93488f784510-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85\" (UID: \"e793c22a-52b4-4605-ab1b-93488f784510\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85"
Jul 28 14:19:32.534874 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.534818    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn6mx\" (UniqueName: \"kubernetes.io/projected/ebaf4287-a41e-478b-b913-ec6e7a66cbb4-kube-api-access-qn6mx\") pod \"volume-data-source-validator-549c6d54cf-j7vw6\" (UID: \"ebaf4287-a41e-478b-b913-ec6e7a66cbb4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7vw6"
Jul 28 14:19:32.636222 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.636149    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn6mx\" (UniqueName: \"kubernetes.io/projected/ebaf4287-a41e-478b-b913-ec6e7a66cbb4-kube-api-access-qn6mx\") pod \"volume-data-source-validator-549c6d54cf-j7vw6\" (UID: \"ebaf4287-a41e-478b-b913-ec6e7a66cbb4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7vw6"
Jul 28 14:19:32.636340 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.636228    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e793c22a-52b4-4605-ab1b-93488f784510-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85\" (UID: \"e793c22a-52b4-4605-ab1b-93488f784510\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85"
Jul 28 14:19:32.636340 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.636252    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvsdd\" (UniqueName: \"kubernetes.io/projected/e793c22a-52b4-4605-ab1b-93488f784510-kube-api-access-jvsdd\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85\" (UID: \"e793c22a-52b4-4605-ab1b-93488f784510\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85"
Jul 28 14:19:32.636340 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.636292    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e793c22a-52b4-4605-ab1b-93488f784510-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85\" (UID: \"e793c22a-52b4-4605-ab1b-93488f784510\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85"
Jul 28 14:19:32.636904 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.636886    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e793c22a-52b4-4605-ab1b-93488f784510-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85\" (UID: \"e793c22a-52b4-4605-ab1b-93488f784510\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85"
Jul 28 14:19:32.639965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.639943    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e793c22a-52b4-4605-ab1b-93488f784510-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85\" (UID: \"e793c22a-52b4-4605-ab1b-93488f784510\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85"
Jul 28 14:19:32.645343 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.645309    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn6mx\" (UniqueName: \"kubernetes.io/projected/ebaf4287-a41e-478b-b913-ec6e7a66cbb4-kube-api-access-qn6mx\") pod \"volume-data-source-validator-549c6d54cf-j7vw6\" (UID: \"ebaf4287-a41e-478b-b913-ec6e7a66cbb4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7vw6"
Jul 28 14:19:32.645452 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.645357    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvsdd\" (UniqueName: \"kubernetes.io/projected/e793c22a-52b4-4605-ab1b-93488f784510-kube-api-access-jvsdd\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85\" (UID: \"e793c22a-52b4-4605-ab1b-93488f784510\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85"
Jul 28 14:19:32.695854 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.695809    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85"
Jul 28 14:19:32.716027 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.715995    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7vw6"
Jul 28 14:19:32.853541 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.853491    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85"]
Jul 28 14:19:32.864758 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:19:32.864726    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode793c22a_52b4_4605_ab1b_93488f784510.slice/crio-a4aabe067add29051f197f62cabfe4310ac94be63105a2d343f36fbe45701a0d WatchSource:0}: Error finding container a4aabe067add29051f197f62cabfe4310ac94be63105a2d343f36fbe45701a0d: Status 404 returned error can't find the container with id a4aabe067add29051f197f62cabfe4310ac94be63105a2d343f36fbe45701a0d
Jul 28 14:19:32.901376 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:32.901305    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7vw6"]
Jul 28 14:19:32.904266 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:19:32.904237    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebaf4287_a41e_478b_b913_ec6e7a66cbb4.slice/crio-df6974538ad1ade2142d8402ed3bf299f0db6363fa4f0e15477647baa26e9a94 WatchSource:0}: Error finding container df6974538ad1ade2142d8402ed3bf299f0db6363fa4f0e15477647baa26e9a94: Status 404 returned error can't find the container with id df6974538ad1ade2142d8402ed3bf299f0db6363fa4f0e15477647baa26e9a94
Jul 28 14:19:33.078708 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:33.078664    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85" event={"ID":"e793c22a-52b4-4605-ab1b-93488f784510","Type":"ContainerStarted","Data":"a4aabe067add29051f197f62cabfe4310ac94be63105a2d343f36fbe45701a0d"}
Jul 28 14:19:33.079602 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:33.079581    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7vw6" event={"ID":"ebaf4287-a41e-478b-b913-ec6e7a66cbb4","Type":"ContainerStarted","Data":"df6974538ad1ade2142d8402ed3bf299f0db6363fa4f0e15477647baa26e9a94"}
Jul 28 14:19:33.847127 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:33.847070    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls\") pod \"dns-default-hkt4c\" (UID: \"8dc38f88-df97-4d9f-8718-306934c61e2b\") " pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:33.847615 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:33.847217    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 14:19:33.847615 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:33.847304    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls podName:8dc38f88-df97-4d9f-8718-306934c61e2b nodeName:}" failed. No retries permitted until 2026-07-28 14:19:41.847277866 +0000 UTC m=+51.648820077 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls") pod "dns-default-hkt4c" (UID: "8dc38f88-df97-4d9f-8718-306934c61e2b") : secret "dns-default-metrics-tls" not found
Jul 28 14:19:34.451626 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:34.451581    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:34.451626 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:34.451621    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:34.451828 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:34.451706    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 14:19:34.451828 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:34.451772    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-rbac-proxy-cm podName:d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d nodeName:}" failed. No retries permitted until 2026-07-28 14:19:42.451757182 +0000 UTC m=+52.253299376 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cbx4k" (UID: "d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d") : configmap "kube-rbac-proxy" not found
Jul 28 14:19:34.451828 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:34.451718    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:34.451924 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:34.451845    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls podName:d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d nodeName:}" failed. No retries permitted until 2026-07-28 14:19:42.451829633 +0000 UTC m=+52.253371848 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cbx4k" (UID: "d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d") : secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:36.090000 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:36.089961    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7vw6" event={"ID":"ebaf4287-a41e-478b-b913-ec6e7a66cbb4","Type":"ContainerStarted","Data":"30349a2ade4d119c6e4a14162e9696eb8a9bb200fd6272e13a3168056d8cc1bf"}
Jul 28 14:19:36.091488 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:36.091405    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85" event={"ID":"e793c22a-52b4-4605-ab1b-93488f784510","Type":"ContainerStarted","Data":"1a586a43e83d3e6d3fec0f5ea06f93503709a99ee704a75af56d6e1f659f026a"}
Jul 28 14:19:36.106654 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:36.106599    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7vw6" podStartSLOduration=1.938510613 podStartE2EDuration="4.106584826s" totalImagesPullingTime="2.168074213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:36.105746888 +0000 UTC m=+45.907289103" watchObservedRunningTime="2026-07-28 14:19:36.106584826 +0000 UTC m=+45.908127019"
Jul 28 14:19:36.128007 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:36.127945    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85" podStartSLOduration=1.69459899 podStartE2EDuration="4.12792354s" totalImagesPullingTime="2.43332455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:36.126239182 +0000 UTC m=+45.927781398" watchObservedRunningTime="2026-07-28 14:19:36.12792354 +0000 UTC m=+45.929465756"
Jul 28 14:19:36.362032 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:36.362004    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-5vtv4"]
Jul 28 14:19:36.396939 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:36.396899    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-5vtv4"]
Jul 28 14:19:36.397115 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:36.397022    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-5vtv4"
Jul 28 14:19:36.399817 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:36.399793    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 28 14:19:36.400012 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:36.399890    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 28 14:19:36.400012 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:36.399935    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-clz64\""
Jul 28 14:19:36.469709 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:36.469670    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wzlb\" (UniqueName: \"kubernetes.io/projected/0e4d2302-9c72-49d6-a3d4-5b568cafe842-kube-api-access-7wzlb\") pod \"migrator-6865c9967-5vtv4\" (UID: \"0e4d2302-9c72-49d6-a3d4-5b568cafe842\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-5vtv4"
Jul 28 14:19:36.570064 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:36.570021    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wzlb\" (UniqueName: \"kubernetes.io/projected/0e4d2302-9c72-49d6-a3d4-5b568cafe842-kube-api-access-7wzlb\") pod \"migrator-6865c9967-5vtv4\" (UID: \"0e4d2302-9c72-49d6-a3d4-5b568cafe842\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-5vtv4"
Jul 28 14:19:36.578496 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:36.578475    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wzlb\" (UniqueName: \"kubernetes.io/projected/0e4d2302-9c72-49d6-a3d4-5b568cafe842-kube-api-access-7wzlb\") pod \"migrator-6865c9967-5vtv4\" (UID: \"0e4d2302-9c72-49d6-a3d4-5b568cafe842\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-5vtv4"
Jul 28 14:19:36.706912 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:36.706827    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-5vtv4"
Jul 28 14:19:36.845103 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:36.845058    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-5vtv4"]
Jul 28 14:19:36.848297 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:19:36.848271    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e4d2302_9c72_49d6_a3d4_5b568cafe842.slice/crio-cf0099d5465d71b1f57fd9944eff7ba16f93bd0ddbb752438d92be8caa8307a0 WatchSource:0}: Error finding container cf0099d5465d71b1f57fd9944eff7ba16f93bd0ddbb752438d92be8caa8307a0: Status 404 returned error can't find the container with id cf0099d5465d71b1f57fd9944eff7ba16f93bd0ddbb752438d92be8caa8307a0
Jul 28 14:19:37.094851 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:37.094754    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-5vtv4" event={"ID":"0e4d2302-9c72-49d6-a3d4-5b568cafe842","Type":"ContainerStarted","Data":"cf0099d5465d71b1f57fd9944eff7ba16f93bd0ddbb752438d92be8caa8307a0"}
Jul 28 14:19:39.101021 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:39.100984    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-5vtv4" event={"ID":"0e4d2302-9c72-49d6-a3d4-5b568cafe842","Type":"ContainerStarted","Data":"a801f60802cf985edc7f242339d441a9d0bfa5ef191ada890a72247646dc5d36"}
Jul 28 14:19:39.101436 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:39.101050    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-5vtv4" event={"ID":"0e4d2302-9c72-49d6-a3d4-5b568cafe842","Type":"ContainerStarted","Data":"91571be574b19fbbc8f6d1036c4faa56a563571dec5136d8d01f10a13c011644"}
Jul 28 14:19:39.118432 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:39.118379    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-5vtv4" podStartSLOduration=1.165052583 podStartE2EDuration="3.118362319s" totalImagesPullingTime="1.953309736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:19:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:39.118243886 +0000 UTC m=+48.919786101" watchObservedRunningTime="2026-07-28 14:19:39.118362319 +0000 UTC m=+48.919904534"
Jul 28 14:19:41.909974 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:41.909932    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls\") pod \"dns-default-hkt4c\" (UID: \"8dc38f88-df97-4d9f-8718-306934c61e2b\") " pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:41.910405 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:41.910082    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 14:19:41.910405 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:41.910170    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls podName:8dc38f88-df97-4d9f-8718-306934c61e2b nodeName:}" failed. No retries permitted until 2026-07-28 14:19:57.910152468 +0000 UTC m=+67.711694679 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls") pod "dns-default-hkt4c" (UID: "8dc38f88-df97-4d9f-8718-306934c61e2b") : secret "dns-default-metrics-tls" not found
Jul 28 14:19:42.515134 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:42.515077    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:42.515303 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:42.515140    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:42.515303 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:42.515262    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:42.515380 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:19:42.515330    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls podName:d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d nodeName:}" failed. No retries permitted until 2026-07-28 14:19:58.51531191 +0000 UTC m=+68.316854103 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cbx4k" (UID: "d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d") : secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:42.515731 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:42.515708    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:52.043072 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:52.043043    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lr6gc"
Jul 28 14:19:55.509429 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:55.509382    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret\") pod \"global-pull-secret-syncer-d7fhl\" (UID: \"9ffc9cda-254d-4825-bb1a-1872331a8cc9\") " pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:55.509429 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:55.509431    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs\") pod \"network-metrics-daemon-fx9qn\" (UID: \"9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c\") " pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:19:55.512754 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:55.512733    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 14:19:55.512847 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:55.512832    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 14:19:55.522114 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:55.522064    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c-metrics-certs\") pod \"network-metrics-daemon-fx9qn\" (UID: \"9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c\") " pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:19:55.522233 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:55.522150    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ffc9cda-254d-4825-bb1a-1872331a8cc9-original-pull-secret\") pod \"global-pull-secret-syncer-d7fhl\" (UID: \"9ffc9cda-254d-4825-bb1a-1872331a8cc9\") " pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:55.610463 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:55.610427    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlwv6\" (UniqueName: \"kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6\") pod \"network-check-target-lcn77\" (UID: \"dbe349f8-37a2-49ce-b92c-6100612b24b6\") " pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:55.613123 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:55.613088    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 14:19:55.623618 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:55.623594    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 14:19:55.633552 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:55.633529    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlwv6\" (UniqueName: \"kubernetes.io/projected/dbe349f8-37a2-49ce-b92c-6100612b24b6-kube-api-access-vlwv6\") pod \"network-check-target-lcn77\" (UID: \"dbe349f8-37a2-49ce-b92c-6100612b24b6\") " pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:55.703620 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:55.703580    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7fhl"
Jul 28 14:19:55.714140 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:55.714116    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v4vsp\""
Jul 28 14:19:55.720300 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:55.720280    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4gd6j\""
Jul 28 14:19:55.722238 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:55.722224    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:19:55.728036 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:55.728014    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx9qn"
Jul 28 14:19:55.847500 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:55.847421    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d7fhl"]
Jul 28 14:19:55.851306 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:19:55.851274    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ffc9cda_254d_4825_bb1a_1872331a8cc9.slice/crio-354a1674b2e11f58fa49a07d05632a1b1e4f9a6e2f4f9c782b119527fb579155 WatchSource:0}: Error finding container 354a1674b2e11f58fa49a07d05632a1b1e4f9a6e2f4f9c782b119527fb579155: Status 404 returned error can't find the container with id 354a1674b2e11f58fa49a07d05632a1b1e4f9a6e2f4f9c782b119527fb579155
Jul 28 14:19:55.862028 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:55.861993    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lcn77"]
Jul 28 14:19:55.865506 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:19:55.865480    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbe349f8_37a2_49ce_b92c_6100612b24b6.slice/crio-8887a5d6bf85b4ff834d9d9a13ec57869448d0934cbf8d700740848ce85563e1 WatchSource:0}: Error finding container 8887a5d6bf85b4ff834d9d9a13ec57869448d0934cbf8d700740848ce85563e1: Status 404 returned error can't find the container with id 8887a5d6bf85b4ff834d9d9a13ec57869448d0934cbf8d700740848ce85563e1
Jul 28 14:19:55.886234 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:55.886211    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fx9qn"]
Jul 28 14:19:55.889192 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:19:55.889171    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c6c4d1a_f5b9_412e_93bf_c5a1c5026c0c.slice/crio-c20c4a484ccd00715a49d8510d06c0f1e152df7af010171eea601c2b07da7aa6 WatchSource:0}: Error finding container c20c4a484ccd00715a49d8510d06c0f1e152df7af010171eea601c2b07da7aa6: Status 404 returned error can't find the container with id c20c4a484ccd00715a49d8510d06c0f1e152df7af010171eea601c2b07da7aa6
Jul 28 14:19:56.144105 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:56.144066    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d7fhl" event={"ID":"9ffc9cda-254d-4825-bb1a-1872331a8cc9","Type":"ContainerStarted","Data":"354a1674b2e11f58fa49a07d05632a1b1e4f9a6e2f4f9c782b119527fb579155"}
Jul 28 14:19:56.145151 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:56.145121    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fx9qn" event={"ID":"9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c","Type":"ContainerStarted","Data":"c20c4a484ccd00715a49d8510d06c0f1e152df7af010171eea601c2b07da7aa6"}
Jul 28 14:19:56.146166 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:56.146146    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lcn77" event={"ID":"dbe349f8-37a2-49ce-b92c-6100612b24b6","Type":"ContainerStarted","Data":"8887a5d6bf85b4ff834d9d9a13ec57869448d0934cbf8d700740848ce85563e1"}
Jul 28 14:19:57.930111 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:57.930057    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls\") pod \"dns-default-hkt4c\" (UID: \"8dc38f88-df97-4d9f-8718-306934c61e2b\") " pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:57.932706 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:57.932683    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dc38f88-df97-4d9f-8718-306934c61e2b-metrics-tls\") pod \"dns-default-hkt4c\" (UID: \"8dc38f88-df97-4d9f-8718-306934c61e2b\") " pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:58.155273 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:58.155232    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fx9qn" event={"ID":"9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c","Type":"ContainerStarted","Data":"1a8f293149409c689af5f353e2c028506b025a6171841e236b2f1aa887703fe5"}
Jul 28 14:19:58.155461 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:58.155299    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fx9qn" event={"ID":"9c6c4d1a-f5b9-412e-93bf-c5a1c5026c0c","Type":"ContainerStarted","Data":"001e8a663eb0df66d75aa318ec720f61cb27b2a1c4045bb5ff858734764c9e64"}
Jul 28 14:19:58.175484 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:58.175418    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fx9qn" podStartSLOduration=65.990682549 podStartE2EDuration="1m7.175394765s" totalImagesPullingTime="1.184712216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:58.173385983 +0000 UTC m=+67.974928211" watchObservedRunningTime="2026-07-28 14:19:58.175394765 +0000 UTC m=+67.976936981"
Jul 28 14:19:58.179225 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:58.179201    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6f28j\""
Jul 28 14:19:58.187501 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:58.187417    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:19:58.535958 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:58.535878    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:58.538990 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:58.538964    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbx4k\" (UID: \"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d\") " pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:58.766672 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:58.766633    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kr6cm\""
Jul 28 14:19:58.774185 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:58.774146    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cbx4k"
Jul 28 14:19:59.524689 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.524653    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"]
Jul 28 14:19:59.545559 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.545515    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.545744 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.545719    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"]
Jul 28 14:19:59.548431 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.548399    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 28 14:19:59.549371 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.549348    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 28 14:19:59.549494 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.549472    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 14:19:59.549554 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.549521    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 28 14:19:59.549611 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.549552    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 14:19:59.549668 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.549638    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 14:19:59.549708 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.549668    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 28 14:19:59.627865 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.627836    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8g9dz"]
Jul 28 14:19:59.645349 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.645292    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8fea590-3772-43d6-af66-d2836f8de82e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d94475f8d-ztwv6\" (UID: \"e8fea590-3772-43d6-af66-d2836f8de82e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.645349 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.645348    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nndm\" (UniqueName: \"kubernetes.io/projected/e8fea590-3772-43d6-af66-d2836f8de82e-kube-api-access-4nndm\") pod \"cluster-proxy-proxy-agent-6d94475f8d-ztwv6\" (UID: \"e8fea590-3772-43d6-af66-d2836f8de82e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.645569 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.645410    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e8fea590-3772-43d6-af66-d2836f8de82e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d94475f8d-ztwv6\" (UID: \"e8fea590-3772-43d6-af66-d2836f8de82e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.645569 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.645490    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e8fea590-3772-43d6-af66-d2836f8de82e-ca\") pod \"cluster-proxy-proxy-agent-6d94475f8d-ztwv6\" (UID: \"e8fea590-3772-43d6-af66-d2836f8de82e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.645569 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.645542    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e8fea590-3772-43d6-af66-d2836f8de82e-hub\") pod \"cluster-proxy-proxy-agent-6d94475f8d-ztwv6\" (UID: \"e8fea590-3772-43d6-af66-d2836f8de82e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.645717 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.645592    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8fea590-3772-43d6-af66-d2836f8de82e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d94475f8d-ztwv6\" (UID: \"e8fea590-3772-43d6-af66-d2836f8de82e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.647464 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.647401    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8g9dz"]
Jul 28 14:19:59.647612 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.647525    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8g9dz"
Jul 28 14:19:59.650386 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.650362    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 28 14:19:59.650870 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.650658    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-bdcrt\""
Jul 28 14:19:59.691039 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.691007    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d9f895f7-79bkf"]
Jul 28 14:19:59.713465 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.713436    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.717458 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.717428    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lj7bw\""
Jul 28 14:19:59.717625 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.717613    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 14:19:59.717697 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.717645    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 14:19:59.717940 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.717840    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 14:19:59.718081 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.718060    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d9f895f7-79bkf"]
Jul 28 14:19:59.725790 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.725768    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 14:19:59.746351 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.746320    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nndm\" (UniqueName: \"kubernetes.io/projected/e8fea590-3772-43d6-af66-d2836f8de82e-kube-api-access-4nndm\") pod \"cluster-proxy-proxy-agent-6d94475f8d-ztwv6\" (UID: \"e8fea590-3772-43d6-af66-d2836f8de82e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.746518 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.746356    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e8fea590-3772-43d6-af66-d2836f8de82e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d94475f8d-ztwv6\" (UID: \"e8fea590-3772-43d6-af66-d2836f8de82e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.746518 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.746389    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e8fea590-3772-43d6-af66-d2836f8de82e-ca\") pod \"cluster-proxy-proxy-agent-6d94475f8d-ztwv6\" (UID: \"e8fea590-3772-43d6-af66-d2836f8de82e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.746518 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.746444    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e8fea590-3772-43d6-af66-d2836f8de82e-hub\") pod \"cluster-proxy-proxy-agent-6d94475f8d-ztwv6\" (UID: \"e8fea590-3772-43d6-af66-d2836f8de82e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.746518 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.746463    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8fea590-3772-43d6-af66-d2836f8de82e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d94475f8d-ztwv6\" (UID: \"e8fea590-3772-43d6-af66-d2836f8de82e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.746716 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.746553    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1616daad-693b-483b-a760-015508392f41-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-8g9dz\" (UID: \"1616daad-693b-483b-a760-015508392f41\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8g9dz"
Jul 28 14:19:59.746716 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.746596    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8fea590-3772-43d6-af66-d2836f8de82e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d94475f8d-ztwv6\" (UID: \"e8fea590-3772-43d6-af66-d2836f8de82e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.747339 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.747312    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8fea590-3772-43d6-af66-d2836f8de82e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d94475f8d-ztwv6\" (UID: \"e8fea590-3772-43d6-af66-d2836f8de82e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.748943 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.748891    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e8fea590-3772-43d6-af66-d2836f8de82e-ca\") pod \"cluster-proxy-proxy-agent-6d94475f8d-ztwv6\" (UID: \"e8fea590-3772-43d6-af66-d2836f8de82e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.749332 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.749314    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e8fea590-3772-43d6-af66-d2836f8de82e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d94475f8d-ztwv6\" (UID: \"e8fea590-3772-43d6-af66-d2836f8de82e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.749452 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.749434    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8fea590-3772-43d6-af66-d2836f8de82e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d94475f8d-ztwv6\" (UID: \"e8fea590-3772-43d6-af66-d2836f8de82e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.749558 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.749535    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e8fea590-3772-43d6-af66-d2836f8de82e-hub\") pod \"cluster-proxy-proxy-agent-6d94475f8d-ztwv6\" (UID: \"e8fea590-3772-43d6-af66-d2836f8de82e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.756740 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.756711    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nndm\" (UniqueName: \"kubernetes.io/projected/e8fea590-3772-43d6-af66-d2836f8de82e-kube-api-access-4nndm\") pod \"cluster-proxy-proxy-agent-6d94475f8d-ztwv6\" (UID: \"e8fea590-3772-43d6-af66-d2836f8de82e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.847081 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.846983    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b8f57f2-979c-4920-a974-e4de894b8b62-trusted-ca\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.847081 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.847059    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1616daad-693b-483b-a760-015508392f41-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-8g9dz\" (UID: \"1616daad-693b-483b-a760-015508392f41\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8g9dz"
Jul 28 14:19:59.847317 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.847087    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b8f57f2-979c-4920-a974-e4de894b8b62-ca-trust-extracted\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.847317 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.847134    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b8f57f2-979c-4920-a974-e4de894b8b62-registry-certificates\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.847317 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.847161    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b8f57f2-979c-4920-a974-e4de894b8b62-bound-sa-token\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.847317 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.847213    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5s26\" (UniqueName: \"kubernetes.io/projected/9b8f57f2-979c-4920-a974-e4de894b8b62-kube-api-access-f5s26\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.847317 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.847259    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b8f57f2-979c-4920-a974-e4de894b8b62-image-registry-private-configuration\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.847317 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.847301    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b8f57f2-979c-4920-a974-e4de894b8b62-registry-tls\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.847571 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.847395    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b8f57f2-979c-4920-a974-e4de894b8b62-installation-pull-secrets\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.849829 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.849797    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1616daad-693b-483b-a760-015508392f41-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-8g9dz\" (UID: \"1616daad-693b-483b-a760-015508392f41\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8g9dz"
Jul 28 14:19:59.866638 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.866592    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"
Jul 28 14:19:59.948086 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.948046    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b8f57f2-979c-4920-a974-e4de894b8b62-installation-pull-secrets\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.948305 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.948107    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b8f57f2-979c-4920-a974-e4de894b8b62-trusted-ca\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.948305 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.948155    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b8f57f2-979c-4920-a974-e4de894b8b62-ca-trust-extracted\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.948305 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.948190    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b8f57f2-979c-4920-a974-e4de894b8b62-registry-certificates\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.948305 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.948229    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b8f57f2-979c-4920-a974-e4de894b8b62-bound-sa-token\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.948305 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.948265    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5s26\" (UniqueName: \"kubernetes.io/projected/9b8f57f2-979c-4920-a974-e4de894b8b62-kube-api-access-f5s26\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.948578 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.948307    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b8f57f2-979c-4920-a974-e4de894b8b62-image-registry-private-configuration\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.948578 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.948351    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b8f57f2-979c-4920-a974-e4de894b8b62-registry-tls\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.948688 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.948661    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b8f57f2-979c-4920-a974-e4de894b8b62-ca-trust-extracted\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.949520 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.949493    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b8f57f2-979c-4920-a974-e4de894b8b62-trusted-ca\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.949520 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.949502    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b8f57f2-979c-4920-a974-e4de894b8b62-registry-certificates\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.951436 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.951413    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b8f57f2-979c-4920-a974-e4de894b8b62-installation-pull-secrets\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.951527 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.951469    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b8f57f2-979c-4920-a974-e4de894b8b62-registry-tls\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.952240 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.952219    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b8f57f2-979c-4920-a974-e4de894b8b62-image-registry-private-configuration\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.957558 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.957349    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8g9dz"
Jul 28 14:19:59.958476 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.958455    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b8f57f2-979c-4920-a974-e4de894b8b62-bound-sa-token\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:19:59.958594 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:19:59.958574    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5s26\" (UniqueName: \"kubernetes.io/projected/9b8f57f2-979c-4920-a974-e4de894b8b62-kube-api-access-f5s26\") pod \"image-registry-d9f895f7-79bkf\" (UID: \"9b8f57f2-979c-4920-a974-e4de894b8b62\") " pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:20:00.025651 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:00.025615    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:20:00.859177 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:00.858973    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8g9dz"]
Jul 28 14:20:00.920575 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:00.920268    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6"]
Jul 28 14:20:00.922985 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:20:00.922955    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8fea590_3772_43d6_af66_d2836f8de82e.slice/crio-97b8d5690690d67e2a79f2883603054f0105eb521d8ae368d1c3159c31da30e9 WatchSource:0}: Error finding container 97b8d5690690d67e2a79f2883603054f0105eb521d8ae368d1c3159c31da30e9: Status 404 returned error can't find the container with id 97b8d5690690d67e2a79f2883603054f0105eb521d8ae368d1c3159c31da30e9
Jul 28 14:20:01.085885 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:01.085861    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cbx4k"]
Jul 28 14:20:01.088202 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:20:01.088169    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1fe4fd0_9976_4a2a_b23f_bd54da1bc59d.slice/crio-5af2f7232ccb9ed7d2f93f6548b1b3e5bd23ff95341b142715f6dd8256ad2d50 WatchSource:0}: Error finding container 5af2f7232ccb9ed7d2f93f6548b1b3e5bd23ff95341b142715f6dd8256ad2d50: Status 404 returned error can't find the container with id 5af2f7232ccb9ed7d2f93f6548b1b3e5bd23ff95341b142715f6dd8256ad2d50
Jul 28 14:20:01.090370 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:01.090344    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hkt4c"]
Jul 28 14:20:01.094272 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:01.094243    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d9f895f7-79bkf"]
Jul 28 14:20:01.099016 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:20:01.098985    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b8f57f2_979c_4920_a974_e4de894b8b62.slice/crio-1416ffaaa605d5ee209711381ef08b6face097a2cbe0b309d02ed12a99537226 WatchSource:0}: Error finding container 1416ffaaa605d5ee209711381ef08b6face097a2cbe0b309d02ed12a99537226: Status 404 returned error can't find the container with id 1416ffaaa605d5ee209711381ef08b6face097a2cbe0b309d02ed12a99537226
Jul 28 14:20:01.166449 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:01.166418    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hkt4c" event={"ID":"8dc38f88-df97-4d9f-8718-306934c61e2b","Type":"ContainerStarted","Data":"221948c8d56afd7b206279238c1cb0e3deb2b644819a6b37775d15ce82e5da15"}
Jul 28 14:20:01.167590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:01.167562    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8g9dz" event={"ID":"1616daad-693b-483b-a760-015508392f41","Type":"ContainerStarted","Data":"99c5141eff1b7b3ae950eaae99fa57b7c6fc7c807b5795cb99cc0c9c5c6497db"}
Jul 28 14:20:01.169081 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:01.169054    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lcn77" event={"ID":"dbe349f8-37a2-49ce-b92c-6100612b24b6","Type":"ContainerStarted","Data":"dee95266a33eb74d305ca58b6ce8315565909c2c9d7a5b101871696313801d0a"}
Jul 28 14:20:01.169214 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:01.169179    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:20:01.170480 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:01.170410    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6" event={"ID":"e8fea590-3772-43d6-af66-d2836f8de82e","Type":"ContainerStarted","Data":"97b8d5690690d67e2a79f2883603054f0105eb521d8ae368d1c3159c31da30e9"}
Jul 28 14:20:01.171814 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:01.171791    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d7fhl" event={"ID":"9ffc9cda-254d-4825-bb1a-1872331a8cc9","Type":"ContainerStarted","Data":"53737e07843120dd8f67fb8344f603da964b1bb595ec1266207420b321a79516"}
Jul 28 14:20:01.173291 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:01.173272    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d9f895f7-79bkf" event={"ID":"9b8f57f2-979c-4920-a974-e4de894b8b62","Type":"ContainerStarted","Data":"5992d9a3e0f7081af18e1f7511fa2efd3ddf773a7d1f38fcdbb7b1561b6ab6c3"}
Jul 28 14:20:01.173395 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:01.173303    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d9f895f7-79bkf" event={"ID":"9b8f57f2-979c-4920-a974-e4de894b8b62","Type":"ContainerStarted","Data":"1416ffaaa605d5ee209711381ef08b6face097a2cbe0b309d02ed12a99537226"}
Jul 28 14:20:01.173395 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:01.173374    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:20:01.174324 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:01.174302    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cbx4k" event={"ID":"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d","Type":"ContainerStarted","Data":"5af2f7232ccb9ed7d2f93f6548b1b3e5bd23ff95341b142715f6dd8256ad2d50"}
Jul 28 14:20:01.189471 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:01.189425    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lcn77" podStartSLOduration=65.407740998 podStartE2EDuration="1m10.189411943s" totalImagesPullingTime="4.781670945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:01.188883165 +0000 UTC m=+70.990425385" watchObservedRunningTime="2026-07-28 14:20:01.189411943 +0000 UTC m=+70.990954157"
Jul 28 14:20:01.217652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:01.217588    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-d9f895f7-79bkf" podStartSLOduration=2.217567383 podStartE2EDuration="2.217567383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:19:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:01.217072447 +0000 UTC m=+71.018614662" watchObservedRunningTime="2026-07-28 14:20:01.217567383 +0000 UTC m=+71.019109599"
Jul 28 14:20:01.234882 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:01.234834    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-d7fhl" podStartSLOduration=65.429229265 podStartE2EDuration="1m10.234818869s" totalImagesPullingTime="4.805589604s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:01.234660575 +0000 UTC m=+71.036202867" watchObservedRunningTime="2026-07-28 14:20:01.234818869 +0000 UTC m=+71.036361084"
Jul 28 14:20:03.196535 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:03.196491    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8g9dz" event={"ID":"1616daad-693b-483b-a760-015508392f41","Type":"ContainerStarted","Data":"b1b5853e3e2e20a812a1d622e1191ec57f5b4fd48e728d7dde49bbc54b2732d0"}
Jul 28 14:20:03.197741 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:03.197684    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8g9dz"
Jul 28 14:20:03.203595 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:03.203569    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8g9dz"
Jul 28 14:20:03.215339 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:03.214580    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8g9dz" podStartSLOduration=2.3357476249999998 podStartE2EDuration="4.214560713s" totalImagesPullingTime="1.878813088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:19:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:03.213659467 +0000 UTC m=+73.015201685" watchObservedRunningTime="2026-07-28 14:20:03.214560713 +0000 UTC m=+73.016102911"
Jul 28 14:20:09.223587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:09.223511    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cbx4k" event={"ID":"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d","Type":"ContainerStarted","Data":"3d3af497971475eeb5c29cf436458a68edbe247485de275811e657ffe5555334"}
Jul 28 14:20:09.226816 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:09.226681    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hkt4c" event={"ID":"8dc38f88-df97-4d9f-8718-306934c61e2b","Type":"ContainerStarted","Data":"d272799a2a69be6ad6b6a862c3611d97d3a868213ee8b85497df5cb3e5aba5cb"}
Jul 28 14:20:09.226816 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:09.226748    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hkt4c" event={"ID":"8dc38f88-df97-4d9f-8718-306934c61e2b","Type":"ContainerStarted","Data":"c3163eb3f74209a77c4154ec317424526fb229ee6805f7b051f2c353634f11d3"}
Jul 28 14:20:09.226816 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:09.226783    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:20:09.229436 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:09.229401    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6" event={"ID":"e8fea590-3772-43d6-af66-d2836f8de82e","Type":"ContainerStarted","Data":"ca7814f2a2a16e4caa5f7eb04719e5588c7d1f11e62e37c92f30fd00ef41456a"}
Jul 28 14:20:09.229594 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:09.229450    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6" event={"ID":"e8fea590-3772-43d6-af66-d2836f8de82e","Type":"ContainerStarted","Data":"4fdc950c4f208c99fcc4adcb5bd28cd0bd6881700da8ff0beee91e7940f3f35f"}
Jul 28 14:20:09.229594 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:09.229470    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6" event={"ID":"e8fea590-3772-43d6-af66-d2836f8de82e","Type":"ContainerStarted","Data":"5173951ef94e2f2f4e74219cd14ff6e032ffe6466302a831bc146843b1447a43"}
Jul 28 14:20:09.247781 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:09.247493    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hkt4c" podStartSLOduration=36.032148043 podStartE2EDuration="43.247468232s" totalImagesPullingTime="7.215320189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:19:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:09.245295265 +0000 UTC m=+79.046837481" watchObservedRunningTime="2026-07-28 14:20:09.247468232 +0000 UTC m=+79.049010448"
Jul 28 14:20:11.526582 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.526496    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d94475f8d-ztwv6" podStartSLOduration=5.117829731 podStartE2EDuration="12.526471947s" totalImagesPullingTime="7.408642216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:19:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:09.268299399 +0000 UTC m=+79.069841614" watchObservedRunningTime="2026-07-28 14:20:11.526471947 +0000 UTC m=+81.328014165"
Jul 28 14:20:11.527075 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.526711    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pc4z2"]
Jul 28 14:20:11.539897 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.539866    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.542838 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.542811    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 14:20:11.542838 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.542826    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 14:20:11.543335 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.543168    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 14:20:11.543335 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.543189    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 14:20:11.543876 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.543763    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-k2gn6\""
Jul 28 14:20:11.543997 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.543900    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 14:20:11.543997 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.543926    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 14:20:11.641545 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.641512    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-node-exporter-accelerators-collector-config\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.641545 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.641553    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.641781 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.641584    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql29x\" (UniqueName: \"kubernetes.io/projected/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-kube-api-access-ql29x\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.641781 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.641687    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-sys\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.641781 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.641739    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-root\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.641885 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.641789    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-node-exporter-textfile\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.641885 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.641822    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-metrics-client-ca\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.641885 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.641863    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-node-exporter-tls\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.642002 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.641899    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-node-exporter-wtmp\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.743059 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.743021    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-node-exporter-textfile\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.743059 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.743060    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-metrics-client-ca\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.743323 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.743086    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-node-exporter-tls\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.743323 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.743131    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-node-exporter-wtmp\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.743323 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.743160    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-node-exporter-accelerators-collector-config\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.743323 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.743188    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.743323 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.743216    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql29x\" (UniqueName: \"kubernetes.io/projected/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-kube-api-access-ql29x\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.743323 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:20:11.743229    2584 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 14:20:11.743323 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.743245    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-sys\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.743323 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:20:11.743301    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-node-exporter-tls podName:1f42b3d4-b9fa-4e4d-b85d-99d733a93f11 nodeName:}" failed. No retries permitted until 2026-07-28 14:20:12.243276 +0000 UTC m=+82.044818208 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-node-exporter-tls") pod "node-exporter-pc4z2" (UID: "1f42b3d4-b9fa-4e4d-b85d-99d733a93f11") : secret "node-exporter-tls" not found
Jul 28 14:20:11.743323 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.743313    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-sys\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.743323 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.743318    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-root\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.743725 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.743355    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-root\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.743725 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.743358    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-node-exporter-wtmp\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.743725 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.743437    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-node-exporter-textfile\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.743725 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.743650    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-metrics-client-ca\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.743851 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.743803    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-node-exporter-accelerators-collector-config\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.745587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.745561    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:11.757392 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:11.757364    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql29x\" (UniqueName: \"kubernetes.io/projected/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-kube-api-access-ql29x\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:12.242823 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:12.242776    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cbx4k" event={"ID":"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d","Type":"ContainerStarted","Data":"ffc15ca4ccf3c15b4ca535df8b1d6ab375806a730f40aef428ba54470f5c199f"}
Jul 28 14:20:12.247171 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:12.247146    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-node-exporter-tls\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:12.249375 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:12.249354    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f42b3d4-b9fa-4e4d-b85d-99d733a93f11-node-exporter-tls\") pod \"node-exporter-pc4z2\" (UID: \"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11\") " pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:12.451706 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:12.451580    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pc4z2"
Jul 28 14:20:12.464844 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:20:12.464813    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f42b3d4_b9fa_4e4d_b85d_99d733a93f11.slice/crio-3da6fbf49943bf1aa03ee166726d25c27c33a982bb1a1d202a055ddd5c224c39 WatchSource:0}: Error finding container 3da6fbf49943bf1aa03ee166726d25c27c33a982bb1a1d202a055ddd5c224c39: Status 404 returned error can't find the container with id 3da6fbf49943bf1aa03ee166726d25c27c33a982bb1a1d202a055ddd5c224c39
Jul 28 14:20:13.248921 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:13.248838    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pc4z2" event={"ID":"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11","Type":"ContainerStarted","Data":"3da6fbf49943bf1aa03ee166726d25c27c33a982bb1a1d202a055ddd5c224c39"}
Jul 28 14:20:14.255132 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:14.255069    2584 generic.go:356] "Generic (PLEG): container finished" podID="1f42b3d4-b9fa-4e4d-b85d-99d733a93f11" containerID="17c13dfe7f6478ef92a6fb7e7524768c17cbd58082bf7c15ab896f9c084e24b2" exitCode=0
Jul 28 14:20:14.255577 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:14.255160    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pc4z2" event={"ID":"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11","Type":"ContainerDied","Data":"17c13dfe7f6478ef92a6fb7e7524768c17cbd58082bf7c15ab896f9c084e24b2"}
Jul 28 14:20:16.265653 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:16.265607    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pc4z2" event={"ID":"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11","Type":"ContainerStarted","Data":"2b17a917e1bcce9a7380d6738c3ed85b3811f6f6b3447eaa724954da13395780"}
Jul 28 14:20:16.266135 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:16.265710    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pc4z2" event={"ID":"1f42b3d4-b9fa-4e4d-b85d-99d733a93f11","Type":"ContainerStarted","Data":"e63d49f6a31b20961e61ae127df71c588044b195ecceaeb73688860661540dbe"}
Jul 28 14:20:16.267545 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:16.267524    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cbx4k" event={"ID":"d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d","Type":"ContainerStarted","Data":"fc406c15076399ea3ec5f96dffd5472bc51a7833c290dde6ed9805647897467d"}
Jul 28 14:20:16.302328 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:16.302264    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pc4z2" podStartSLOduration=4.326405093 podStartE2EDuration="5.302244036s" totalImagesPullingTime="902.514511ms" totalInitContainerRuntime="73.324432ms" isStatefulPod=true podCreationTimestamp="2026-07-28 14:20:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:16.302215163 +0000 UTC m=+86.103757379" watchObservedRunningTime="2026-07-28 14:20:16.302244036 +0000 UTC m=+86.103786251"
Jul 28 14:20:16.323349 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:16.323289    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cbx4k" podStartSLOduration=36.143736375 podStartE2EDuration="50.323271873s" totalImagesPullingTime="14.179535498s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:19:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:16.321838229 +0000 UTC m=+86.123380435" watchObservedRunningTime="2026-07-28 14:20:16.323271873 +0000 UTC m=+86.124814088"
Jul 28 14:20:19.236676 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:19.236645    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hkt4c"
Jul 28 14:20:20.030306 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:20.030270    2584 patch_prober.go:28] interesting pod/image-registry-d9f895f7-79bkf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 28 14:20:20.030482 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:20.030332    2584 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-d9f895f7-79bkf" podUID="9b8f57f2-979c-4920-a974-e4de894b8b62" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 14:20:22.187121 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:22.187076    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d9f895f7-79bkf"
Jul 28 14:20:32.184688 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:32.184655    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lcn77"
Jul 28 14:20:41.354880 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:41.354842    2584 generic.go:356] "Generic (PLEG): container finished" podID="e793c22a-52b4-4605-ab1b-93488f784510" containerID="1a586a43e83d3e6d3fec0f5ea06f93503709a99ee704a75af56d6e1f659f026a" exitCode=0
Jul 28 14:20:41.355324 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:41.354916    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85" event={"ID":"e793c22a-52b4-4605-ab1b-93488f784510","Type":"ContainerDied","Data":"1a586a43e83d3e6d3fec0f5ea06f93503709a99ee704a75af56d6e1f659f026a"}
Jul 28 14:20:41.355324 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:41.355299    2584 scope.go:117] "RemoveContainer" containerID="1a586a43e83d3e6d3fec0f5ea06f93503709a99ee704a75af56d6e1f659f026a"
Jul 28 14:20:42.360395 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:20:42.360364    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-f9x85" event={"ID":"e793c22a-52b4-4605-ab1b-93488f784510","Type":"ContainerStarted","Data":"08f0df254f1a34cf81c54883840f48be26d0bc0d797c4cf719a1e6aae0416437"}
Jul 28 14:23:29.996524 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:23:29.996488    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-fpr52"]
Jul 28 14:23:29.998504 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:23:29.998488    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fpr52"
Jul 28 14:23:30.000903 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:23:30.000879    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 14:23:30.001034 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:23:30.000905    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xjfw8\""
Jul 28 14:23:30.001034 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:23:30.000905    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 14:23:30.010032 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:23:30.010004    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-fpr52"]
Jul 28 14:23:30.083725 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:23:30.083683    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9121fa3-3d48-4f36-ac10-c007f609da9c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-fpr52\" (UID: \"d9121fa3-3d48-4f36-ac10-c007f609da9c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fpr52"
Jul 28 14:23:30.083913 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:23:30.083745    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd9mt\" (UniqueName: \"kubernetes.io/projected/d9121fa3-3d48-4f36-ac10-c007f609da9c-kube-api-access-hd9mt\") pod \"cert-manager-cainjector-559cdd8c87-fpr52\" (UID: \"d9121fa3-3d48-4f36-ac10-c007f609da9c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fpr52"
Jul 28 14:23:30.185021 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:23:30.184973    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9121fa3-3d48-4f36-ac10-c007f609da9c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-fpr52\" (UID: \"d9121fa3-3d48-4f36-ac10-c007f609da9c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fpr52"
Jul 28 14:23:30.185021 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:23:30.185030    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd9mt\" (UniqueName: \"kubernetes.io/projected/d9121fa3-3d48-4f36-ac10-c007f609da9c-kube-api-access-hd9mt\") pod \"cert-manager-cainjector-559cdd8c87-fpr52\" (UID: \"d9121fa3-3d48-4f36-ac10-c007f609da9c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fpr52"
Jul 28 14:23:30.193959 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:23:30.193928    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9121fa3-3d48-4f36-ac10-c007f609da9c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-fpr52\" (UID: \"d9121fa3-3d48-4f36-ac10-c007f609da9c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fpr52"
Jul 28 14:23:30.194084 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:23:30.193960    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd9mt\" (UniqueName: \"kubernetes.io/projected/d9121fa3-3d48-4f36-ac10-c007f609da9c-kube-api-access-hd9mt\") pod \"cert-manager-cainjector-559cdd8c87-fpr52\" (UID: \"d9121fa3-3d48-4f36-ac10-c007f609da9c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fpr52"
Jul 28 14:23:30.307876 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:23:30.307795    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fpr52"
Jul 28 14:23:30.437386 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:23:30.437351    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-fpr52"]
Jul 28 14:23:30.440867 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:23:30.440839    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9121fa3_3d48_4f36_ac10_c007f609da9c.slice/crio-b15743e30ffb07e226e59bfbd6e87e0432292a246b134b752f6c184f3238e430 WatchSource:0}: Error finding container b15743e30ffb07e226e59bfbd6e87e0432292a246b134b752f6c184f3238e430: Status 404 returned error can't find the container with id b15743e30ffb07e226e59bfbd6e87e0432292a246b134b752f6c184f3238e430
Jul 28 14:23:30.915082 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:23:30.915048    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fpr52" event={"ID":"d9121fa3-3d48-4f36-ac10-c007f609da9c","Type":"ContainerStarted","Data":"b15743e30ffb07e226e59bfbd6e87e0432292a246b134b752f6c184f3238e430"}
Jul 28 14:23:33.927219 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:23:33.927179    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fpr52" event={"ID":"d9121fa3-3d48-4f36-ac10-c007f609da9c","Type":"ContainerStarted","Data":"43c0398e5599e3ee74ae176056da59ca66b4711469453c3d74cba20b25e576d8"}
Jul 28 14:23:33.952429 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:23:33.952375    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fpr52" podStartSLOduration=2.084200911 podStartE2EDuration="4.952360991s" totalImagesPullingTime="2.86816008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:23:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:23:33.951273504 +0000 UTC m=+283.752815718" watchObservedRunningTime="2026-07-28 14:23:33.952360991 +0000 UTC m=+283.753903205"
Jul 28 14:23:50.601978 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:23:50.601938    2584 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 14:24:03.820585 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.820550    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"]
Jul 28 14:24:03.822520 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.822505    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"
Jul 28 14:24:03.826256 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.826226    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 14:24:03.826688 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.826664    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 14:24:03.826785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.826762    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 14:24:03.827723 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.827706    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 14:24:03.828571 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.828554    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-htct2\""
Jul 28 14:24:03.836265 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.836244    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80778efc-8a53-41a7-b6b3-dd27455c0004-tmp\") pod \"opendatahub-operator-controller-manager-65f7956c87-29zbg\" (UID: \"80778efc-8a53-41a7-b6b3-dd27455c0004\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"
Jul 28 14:24:03.836345 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.836292    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/80778efc-8a53-41a7-b6b3-dd27455c0004-manifests\") pod \"opendatahub-operator-controller-manager-65f7956c87-29zbg\" (UID: \"80778efc-8a53-41a7-b6b3-dd27455c0004\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"
Jul 28 14:24:03.836345 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.836313    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfd86\" (UniqueName: \"kubernetes.io/projected/80778efc-8a53-41a7-b6b3-dd27455c0004-kube-api-access-lfd86\") pod \"opendatahub-operator-controller-manager-65f7956c87-29zbg\" (UID: \"80778efc-8a53-41a7-b6b3-dd27455c0004\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"
Jul 28 14:24:03.836415 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.836356    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/80778efc-8a53-41a7-b6b3-dd27455c0004-webhook-cert\") pod \"opendatahub-operator-controller-manager-65f7956c87-29zbg\" (UID: \"80778efc-8a53-41a7-b6b3-dd27455c0004\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"
Jul 28 14:24:03.836415 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.836372    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/80778efc-8a53-41a7-b6b3-dd27455c0004-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65f7956c87-29zbg\" (UID: \"80778efc-8a53-41a7-b6b3-dd27455c0004\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"
Jul 28 14:24:03.843757 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.843734    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"]
Jul 28 14:24:03.937619 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.937581    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80778efc-8a53-41a7-b6b3-dd27455c0004-tmp\") pod \"opendatahub-operator-controller-manager-65f7956c87-29zbg\" (UID: \"80778efc-8a53-41a7-b6b3-dd27455c0004\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"
Jul 28 14:24:03.937788 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.937653    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/80778efc-8a53-41a7-b6b3-dd27455c0004-manifests\") pod \"opendatahub-operator-controller-manager-65f7956c87-29zbg\" (UID: \"80778efc-8a53-41a7-b6b3-dd27455c0004\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"
Jul 28 14:24:03.937788 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.937685    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfd86\" (UniqueName: \"kubernetes.io/projected/80778efc-8a53-41a7-b6b3-dd27455c0004-kube-api-access-lfd86\") pod \"opendatahub-operator-controller-manager-65f7956c87-29zbg\" (UID: \"80778efc-8a53-41a7-b6b3-dd27455c0004\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"
Jul 28 14:24:03.937788 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.937717    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/80778efc-8a53-41a7-b6b3-dd27455c0004-webhook-cert\") pod \"opendatahub-operator-controller-manager-65f7956c87-29zbg\" (UID: \"80778efc-8a53-41a7-b6b3-dd27455c0004\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"
Jul 28 14:24:03.937788 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.937741    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/80778efc-8a53-41a7-b6b3-dd27455c0004-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65f7956c87-29zbg\" (UID: \"80778efc-8a53-41a7-b6b3-dd27455c0004\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"
Jul 28 14:24:03.938132 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.938087    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/80778efc-8a53-41a7-b6b3-dd27455c0004-manifests\") pod \"opendatahub-operator-controller-manager-65f7956c87-29zbg\" (UID: \"80778efc-8a53-41a7-b6b3-dd27455c0004\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"
Jul 28 14:24:03.940191 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.940159    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80778efc-8a53-41a7-b6b3-dd27455c0004-tmp\") pod \"opendatahub-operator-controller-manager-65f7956c87-29zbg\" (UID: \"80778efc-8a53-41a7-b6b3-dd27455c0004\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"
Jul 28 14:24:03.940297 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.940240    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/80778efc-8a53-41a7-b6b3-dd27455c0004-webhook-cert\") pod \"opendatahub-operator-controller-manager-65f7956c87-29zbg\" (UID: \"80778efc-8a53-41a7-b6b3-dd27455c0004\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"
Jul 28 14:24:03.940353 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.940312    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/80778efc-8a53-41a7-b6b3-dd27455c0004-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65f7956c87-29zbg\" (UID: \"80778efc-8a53-41a7-b6b3-dd27455c0004\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"
Jul 28 14:24:03.956108 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:03.956068    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfd86\" (UniqueName: \"kubernetes.io/projected/80778efc-8a53-41a7-b6b3-dd27455c0004-kube-api-access-lfd86\") pod \"opendatahub-operator-controller-manager-65f7956c87-29zbg\" (UID: \"80778efc-8a53-41a7-b6b3-dd27455c0004\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"
Jul 28 14:24:04.132960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:04.132925    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"
Jul 28 14:24:04.273974 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:04.273946    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"]
Jul 28 14:24:04.277608 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:24:04.277580    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80778efc_8a53_41a7_b6b3_dd27455c0004.slice/crio-ca7623195554152477baa13236858b29cf03af33b64439f4915d8095cabaa9d3 WatchSource:0}: Error finding container ca7623195554152477baa13236858b29cf03af33b64439f4915d8095cabaa9d3: Status 404 returned error can't find the container with id ca7623195554152477baa13236858b29cf03af33b64439f4915d8095cabaa9d3
Jul 28 14:24:04.279416 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:04.279402    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 14:24:05.045257 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:05.045221    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg" event={"ID":"80778efc-8a53-41a7-b6b3-dd27455c0004","Type":"ContainerStarted","Data":"ca7623195554152477baa13236858b29cf03af33b64439f4915d8095cabaa9d3"}
Jul 28 14:24:08.059231 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:08.059138    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg" event={"ID":"80778efc-8a53-41a7-b6b3-dd27455c0004","Type":"ContainerStarted","Data":"80a9902a8e812aa8f945c363b3593358d7d3ed32b8adb634607a43d354b97d00"}
Jul 28 14:24:09.063937 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:09.063853    2584 generic.go:356] "Generic (PLEG): container finished" podID="80778efc-8a53-41a7-b6b3-dd27455c0004" containerID="80a9902a8e812aa8f945c363b3593358d7d3ed32b8adb634607a43d354b97d00" exitCode=0
Jul 28 14:24:09.063937 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:09.063910    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg" event={"ID":"80778efc-8a53-41a7-b6b3-dd27455c0004","Type":"ContainerDied","Data":"80a9902a8e812aa8f945c363b3593358d7d3ed32b8adb634607a43d354b97d00"}
Jul 28 14:24:10.070745 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:10.070704    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg" event={"ID":"80778efc-8a53-41a7-b6b3-dd27455c0004","Type":"ContainerStarted","Data":"a5b29f86b11fe7f029c17891440f563d9348c53b2f9f80b421c74841cb79db35"}
Jul 28 14:24:10.071257 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:10.070819    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"
Jul 28 14:24:10.097518 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:10.097465    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg" podStartSLOduration=2.202502987 podStartE2EDuration="7.097443211s" totalImagesPullingTime="3.858407526s" totalInitContainerRuntime="1.036532698s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:10.095327453 +0000 UTC m=+319.896869670" watchObservedRunningTime="2026-07-28 14:24:10.097443211 +0000 UTC m=+319.898985427"
Jul 28 14:24:21.075708 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:21.075674    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-29zbg"
Jul 28 14:24:24.252338 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.252301    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-65c4fff746-6ksdx"]
Jul 28 14:24:24.259346 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.259320    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65c4fff746-6ksdx"
Jul 28 14:24:24.262741 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.262710    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 14:24:24.262902 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.262884    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 14:24:24.263889 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.263870    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 14:24:24.264038 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.263901    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 14:24:24.280795 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.280765    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65c4fff746-6ksdx"]
Jul 28 14:24:24.292570 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.292535    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e108199-9380-42ba-8d02-8361be3b37ad-tls-certs\") pod \"kube-auth-proxy-65c4fff746-6ksdx\" (UID: \"4e108199-9380-42ba-8d02-8361be3b37ad\") " pod="openshift-ingress/kube-auth-proxy-65c4fff746-6ksdx"
Jul 28 14:24:24.292570 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.292569    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e108199-9380-42ba-8d02-8361be3b37ad-tmp\") pod \"kube-auth-proxy-65c4fff746-6ksdx\" (UID: \"4e108199-9380-42ba-8d02-8361be3b37ad\") " pod="openshift-ingress/kube-auth-proxy-65c4fff746-6ksdx"
Jul 28 14:24:24.292791 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.292664    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx5nw\" (UniqueName: \"kubernetes.io/projected/4e108199-9380-42ba-8d02-8361be3b37ad-kube-api-access-zx5nw\") pod \"kube-auth-proxy-65c4fff746-6ksdx\" (UID: \"4e108199-9380-42ba-8d02-8361be3b37ad\") " pod="openshift-ingress/kube-auth-proxy-65c4fff746-6ksdx"
Jul 28 14:24:24.394148 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.394083    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e108199-9380-42ba-8d02-8361be3b37ad-tls-certs\") pod \"kube-auth-proxy-65c4fff746-6ksdx\" (UID: \"4e108199-9380-42ba-8d02-8361be3b37ad\") " pod="openshift-ingress/kube-auth-proxy-65c4fff746-6ksdx"
Jul 28 14:24:24.394148 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.394146    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e108199-9380-42ba-8d02-8361be3b37ad-tmp\") pod \"kube-auth-proxy-65c4fff746-6ksdx\" (UID: \"4e108199-9380-42ba-8d02-8361be3b37ad\") " pod="openshift-ingress/kube-auth-proxy-65c4fff746-6ksdx"
Jul 28 14:24:24.394433 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.394185    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx5nw\" (UniqueName: \"kubernetes.io/projected/4e108199-9380-42ba-8d02-8361be3b37ad-kube-api-access-zx5nw\") pod \"kube-auth-proxy-65c4fff746-6ksdx\" (UID: \"4e108199-9380-42ba-8d02-8361be3b37ad\") " pod="openshift-ingress/kube-auth-proxy-65c4fff746-6ksdx"
Jul 28 14:24:24.396707 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.396679    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e108199-9380-42ba-8d02-8361be3b37ad-tls-certs\") pod \"kube-auth-proxy-65c4fff746-6ksdx\" (UID: \"4e108199-9380-42ba-8d02-8361be3b37ad\") " pod="openshift-ingress/kube-auth-proxy-65c4fff746-6ksdx"
Jul 28 14:24:24.396829 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.396738    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e108199-9380-42ba-8d02-8361be3b37ad-tmp\") pod \"kube-auth-proxy-65c4fff746-6ksdx\" (UID: \"4e108199-9380-42ba-8d02-8361be3b37ad\") " pod="openshift-ingress/kube-auth-proxy-65c4fff746-6ksdx"
Jul 28 14:24:24.410625 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.410594    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx5nw\" (UniqueName: \"kubernetes.io/projected/4e108199-9380-42ba-8d02-8361be3b37ad-kube-api-access-zx5nw\") pod \"kube-auth-proxy-65c4fff746-6ksdx\" (UID: \"4e108199-9380-42ba-8d02-8361be3b37ad\") " pod="openshift-ingress/kube-auth-proxy-65c4fff746-6ksdx"
Jul 28 14:24:24.524954 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.524878    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fwf8p"]
Jul 28 14:24:24.528355 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.528339    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fwf8p"
Jul 28 14:24:24.530995 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.530975    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 28 14:24:24.531170 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.531014    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-lgb2n\""
Jul 28 14:24:24.548584 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.548552    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fwf8p"]
Jul 28 14:24:24.569241 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.569204    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65c4fff746-6ksdx"
Jul 28 14:24:24.595581 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.595546    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcwvj\" (UniqueName: \"kubernetes.io/projected/e360876b-132e-4a38-bbeb-d6c87c9b8548-kube-api-access-xcwvj\") pod \"odh-model-controller-7998c67dc4-fwf8p\" (UID: \"e360876b-132e-4a38-bbeb-d6c87c9b8548\") " pod="opendatahub/odh-model-controller-7998c67dc4-fwf8p"
Jul 28 14:24:24.595736 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.595591    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e360876b-132e-4a38-bbeb-d6c87c9b8548-cert\") pod \"odh-model-controller-7998c67dc4-fwf8p\" (UID: \"e360876b-132e-4a38-bbeb-d6c87c9b8548\") " pod="opendatahub/odh-model-controller-7998c67dc4-fwf8p"
Jul 28 14:24:24.696415 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.696385    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcwvj\" (UniqueName: \"kubernetes.io/projected/e360876b-132e-4a38-bbeb-d6c87c9b8548-kube-api-access-xcwvj\") pod \"odh-model-controller-7998c67dc4-fwf8p\" (UID: \"e360876b-132e-4a38-bbeb-d6c87c9b8548\") " pod="opendatahub/odh-model-controller-7998c67dc4-fwf8p"
Jul 28 14:24:24.696415 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.696422    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e360876b-132e-4a38-bbeb-d6c87c9b8548-cert\") pod \"odh-model-controller-7998c67dc4-fwf8p\" (UID: \"e360876b-132e-4a38-bbeb-d6c87c9b8548\") " pod="opendatahub/odh-model-controller-7998c67dc4-fwf8p"
Jul 28 14:24:24.696671 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:24:24.696534    2584 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 28 14:24:24.696671 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:24:24.696589    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e360876b-132e-4a38-bbeb-d6c87c9b8548-cert podName:e360876b-132e-4a38-bbeb-d6c87c9b8548 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:25.196573794 +0000 UTC m=+334.998115999 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e360876b-132e-4a38-bbeb-d6c87c9b8548-cert") pod "odh-model-controller-7998c67dc4-fwf8p" (UID: "e360876b-132e-4a38-bbeb-d6c87c9b8548") : secret "odh-model-controller-webhook-cert" not found
Jul 28 14:24:24.708357 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.708323    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcwvj\" (UniqueName: \"kubernetes.io/projected/e360876b-132e-4a38-bbeb-d6c87c9b8548-kube-api-access-xcwvj\") pod \"odh-model-controller-7998c67dc4-fwf8p\" (UID: \"e360876b-132e-4a38-bbeb-d6c87c9b8548\") " pod="opendatahub/odh-model-controller-7998c67dc4-fwf8p"
Jul 28 14:24:24.716997 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:24.716977    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65c4fff746-6ksdx"]
Jul 28 14:24:24.719414 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:24:24.719389    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e108199_9380_42ba_8d02_8361be3b37ad.slice/crio-6ebfddad897bc591dcd3b9b8863b9092170b34fc13fd63f758ffd9b101c9e06e WatchSource:0}: Error finding container 6ebfddad897bc591dcd3b9b8863b9092170b34fc13fd63f758ffd9b101c9e06e: Status 404 returned error can't find the container with id 6ebfddad897bc591dcd3b9b8863b9092170b34fc13fd63f758ffd9b101c9e06e
Jul 28 14:24:25.127063 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.127028    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65c4fff746-6ksdx" event={"ID":"4e108199-9380-42ba-8d02-8361be3b37ad","Type":"ContainerStarted","Data":"6ebfddad897bc591dcd3b9b8863b9092170b34fc13fd63f758ffd9b101c9e06e"}
Jul 28 14:24:25.201525 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.201491    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e360876b-132e-4a38-bbeb-d6c87c9b8548-cert\") pod \"odh-model-controller-7998c67dc4-fwf8p\" (UID: \"e360876b-132e-4a38-bbeb-d6c87c9b8548\") " pod="opendatahub/odh-model-controller-7998c67dc4-fwf8p"
Jul 28 14:24:25.203945 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.203915    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e360876b-132e-4a38-bbeb-d6c87c9b8548-cert\") pod \"odh-model-controller-7998c67dc4-fwf8p\" (UID: \"e360876b-132e-4a38-bbeb-d6c87c9b8548\") " pod="opendatahub/odh-model-controller-7998c67dc4-fwf8p"
Jul 28 14:24:25.439035 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.438935    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fwf8p"
Jul 28 14:24:25.568701 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.568659    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dbm5c"]
Jul 28 14:24:25.572513 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.572479    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dbm5c"
Jul 28 14:24:25.575241 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.575215    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 28 14:24:25.584910 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.584883    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dbm5c"]
Jul 28 14:24:25.600943 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.600914    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fwf8p"]
Jul 28 14:24:25.604273 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:24:25.604241    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode360876b_132e_4a38_bbeb_d6c87c9b8548.slice/crio-a6718091b74c4af48b8df9f74d48fd39ef19a77a1f2f1079cb79667047c42165 WatchSource:0}: Error finding container a6718091b74c4af48b8df9f74d48fd39ef19a77a1f2f1079cb79667047c42165: Status 404 returned error can't find the container with id a6718091b74c4af48b8df9f74d48fd39ef19a77a1f2f1079cb79667047c42165
Jul 28 14:24:25.605362 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.605337    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktkz7\" (UniqueName: \"kubernetes.io/projected/14c17f23-5dac-413e-9df4-bf19cfa8c4ef-kube-api-access-ktkz7\") pod \"model-serving-api-54d89bd5fc-dbm5c\" (UID: \"14c17f23-5dac-413e-9df4-bf19cfa8c4ef\") " pod="opendatahub/model-serving-api-54d89bd5fc-dbm5c"
Jul 28 14:24:25.605481 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.605414    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14c17f23-5dac-413e-9df4-bf19cfa8c4ef-tls-certs\") pod \"model-serving-api-54d89bd5fc-dbm5c\" (UID: \"14c17f23-5dac-413e-9df4-bf19cfa8c4ef\") " pod="opendatahub/model-serving-api-54d89bd5fc-dbm5c"
Jul 28 14:24:25.706948 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.706856    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktkz7\" (UniqueName: \"kubernetes.io/projected/14c17f23-5dac-413e-9df4-bf19cfa8c4ef-kube-api-access-ktkz7\") pod \"model-serving-api-54d89bd5fc-dbm5c\" (UID: \"14c17f23-5dac-413e-9df4-bf19cfa8c4ef\") " pod="opendatahub/model-serving-api-54d89bd5fc-dbm5c"
Jul 28 14:24:25.707145 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.706952    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14c17f23-5dac-413e-9df4-bf19cfa8c4ef-tls-certs\") pod \"model-serving-api-54d89bd5fc-dbm5c\" (UID: \"14c17f23-5dac-413e-9df4-bf19cfa8c4ef\") " pod="opendatahub/model-serving-api-54d89bd5fc-dbm5c"
Jul 28 14:24:25.707145 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:24:25.707073    2584 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 28 14:24:25.707263 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:24:25.707152    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14c17f23-5dac-413e-9df4-bf19cfa8c4ef-tls-certs podName:14c17f23-5dac-413e-9df4-bf19cfa8c4ef nodeName:}" failed. No retries permitted until 2026-07-28 14:24:26.207132998 +0000 UTC m=+336.008675191 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/14c17f23-5dac-413e-9df4-bf19cfa8c4ef-tls-certs") pod "model-serving-api-54d89bd5fc-dbm5c" (UID: "14c17f23-5dac-413e-9df4-bf19cfa8c4ef") : secret "model-serving-api-tls" not found
Jul 28 14:24:25.720722 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.720688    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktkz7\" (UniqueName: \"kubernetes.io/projected/14c17f23-5dac-413e-9df4-bf19cfa8c4ef-kube-api-access-ktkz7\") pod \"model-serving-api-54d89bd5fc-dbm5c\" (UID: \"14c17f23-5dac-413e-9df4-bf19cfa8c4ef\") " pod="opendatahub/model-serving-api-54d89bd5fc-dbm5c"
Jul 28 14:24:25.828142 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.828108    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"]
Jul 28 14:24:25.831444 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.831418    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:25.833748 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.833725    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 28 14:24:25.833878 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.833726    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 28 14:24:25.833878 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.833840    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 28 14:24:25.834157 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.834141    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 14:24:25.834422 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.834408    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-d4pvr\""
Jul 28 14:24:25.845027 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.844965    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"]
Jul 28 14:24:25.909073 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.909040    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c00eca47-39ef-4e49-9497-316dbee851dd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:25.909073 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.909077    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c00eca47-39ef-4e49-9497-316dbee851dd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:25.909287 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.909112    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c00eca47-39ef-4e49-9497-316dbee851dd-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:25.909287 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.909193    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c00eca47-39ef-4e49-9497-316dbee851dd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:25.909287 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.909227    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c00eca47-39ef-4e49-9497-316dbee851dd-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:25.909287 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.909250    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk9cg\" (UniqueName: \"kubernetes.io/projected/c00eca47-39ef-4e49-9497-316dbee851dd-kube-api-access-gk9cg\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:25.909428 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:25.909357    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c00eca47-39ef-4e49-9497-316dbee851dd-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:26.010532 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.010430    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c00eca47-39ef-4e49-9497-316dbee851dd-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:26.010532 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.010502    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c00eca47-39ef-4e49-9497-316dbee851dd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:26.010764 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.010536    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c00eca47-39ef-4e49-9497-316dbee851dd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:26.010764 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.010563    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c00eca47-39ef-4e49-9497-316dbee851dd-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:26.010764 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.010603    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c00eca47-39ef-4e49-9497-316dbee851dd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:26.010764 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.010626    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c00eca47-39ef-4e49-9497-316dbee851dd-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:26.010764 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.010649    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk9cg\" (UniqueName: \"kubernetes.io/projected/c00eca47-39ef-4e49-9497-316dbee851dd-kube-api-access-gk9cg\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:26.011824 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.011797    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c00eca47-39ef-4e49-9497-316dbee851dd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:26.013331 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.013299    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c00eca47-39ef-4e49-9497-316dbee851dd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:26.013469 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.013367    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c00eca47-39ef-4e49-9497-316dbee851dd-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:26.013469 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.013374    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c00eca47-39ef-4e49-9497-316dbee851dd-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:26.013579 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.013501    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c00eca47-39ef-4e49-9497-316dbee851dd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:26.019357 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.019332    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk9cg\" (UniqueName: \"kubernetes.io/projected/c00eca47-39ef-4e49-9497-316dbee851dd-kube-api-access-gk9cg\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:26.019493 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.019438    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c00eca47-39ef-4e49-9497-316dbee851dd-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-gr8wt\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:26.133733 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.133700    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fwf8p" event={"ID":"e360876b-132e-4a38-bbeb-d6c87c9b8548","Type":"ContainerStarted","Data":"a6718091b74c4af48b8df9f74d48fd39ef19a77a1f2f1079cb79667047c42165"}
Jul 28 14:24:26.142913 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.142877    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:26.212177 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.212060    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14c17f23-5dac-413e-9df4-bf19cfa8c4ef-tls-certs\") pod \"model-serving-api-54d89bd5fc-dbm5c\" (UID: \"14c17f23-5dac-413e-9df4-bf19cfa8c4ef\") " pod="opendatahub/model-serving-api-54d89bd5fc-dbm5c"
Jul 28 14:24:26.215016 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.214951    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14c17f23-5dac-413e-9df4-bf19cfa8c4ef-tls-certs\") pod \"model-serving-api-54d89bd5fc-dbm5c\" (UID: \"14c17f23-5dac-413e-9df4-bf19cfa8c4ef\") " pod="opendatahub/model-serving-api-54d89bd5fc-dbm5c"
Jul 28 14:24:26.303143 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.303054    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"]
Jul 28 14:24:26.306266 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:24:26.306220    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc00eca47_39ef_4e49_9497_316dbee851dd.slice/crio-d645f6ce5f0add53132e74eb7a0282ad8ad54d98fc4df31b652daeea000ce423 WatchSource:0}: Error finding container d645f6ce5f0add53132e74eb7a0282ad8ad54d98fc4df31b652daeea000ce423: Status 404 returned error can't find the container with id d645f6ce5f0add53132e74eb7a0282ad8ad54d98fc4df31b652daeea000ce423
Jul 28 14:24:26.488732 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.488690    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dbm5c"
Jul 28 14:24:26.653559 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:26.653527    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dbm5c"]
Jul 28 14:24:26.657161 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:24:26.657131    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14c17f23_5dac_413e_9df4_bf19cfa8c4ef.slice/crio-11816a43f73c55ec0c1d6c9c9e8240d891308693994e532e1ebb429671ad3598 WatchSource:0}: Error finding container 11816a43f73c55ec0c1d6c9c9e8240d891308693994e532e1ebb429671ad3598: Status 404 returned error can't find the container with id 11816a43f73c55ec0c1d6c9c9e8240d891308693994e532e1ebb429671ad3598
Jul 28 14:24:27.142677 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:27.142549    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dbm5c" event={"ID":"14c17f23-5dac-413e-9df4-bf19cfa8c4ef","Type":"ContainerStarted","Data":"11816a43f73c55ec0c1d6c9c9e8240d891308693994e532e1ebb429671ad3598"}
Jul 28 14:24:27.144747 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:27.144717    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt" event={"ID":"c00eca47-39ef-4e49-9497-316dbee851dd","Type":"ContainerStarted","Data":"d645f6ce5f0add53132e74eb7a0282ad8ad54d98fc4df31b652daeea000ce423"}
Jul 28 14:24:28.155125 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:28.154621    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65c4fff746-6ksdx" event={"ID":"4e108199-9380-42ba-8d02-8361be3b37ad","Type":"ContainerStarted","Data":"f514db5e60a229a76cfbacc653ee1781987de5ff29f95d6e086cc143afb82805"}
Jul 28 14:24:28.180121 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:28.179007    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-65c4fff746-6ksdx" podStartSLOduration=1.132531277 podStartE2EDuration="4.178984279s" totalImagesPullingTime="3.046453002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:28.177077624 +0000 UTC m=+337.978619838" watchObservedRunningTime="2026-07-28 14:24:28.178984279 +0000 UTC m=+337.980526496"
Jul 28 14:24:31.051236 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:31.051179    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:24:31.051686 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:31.051296    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:24:31.171847 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:31.171810    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt" event={"ID":"c00eca47-39ef-4e49-9497-316dbee851dd","Type":"ContainerStarted","Data":"3aab80f8f4a0f7f0e88f88105d2cf007887cd5c509ce0c7b130990339b20d7d8"}
Jul 28 14:24:31.172188 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:31.172135    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:31.173858 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:31.173782    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fwf8p" event={"ID":"e360876b-132e-4a38-bbeb-d6c87c9b8548","Type":"ContainerStarted","Data":"0b126fa59963450fa19d3f12d5a1b27bf7bc2101d41a8b4cd64aebc2f3eceac4"}
Jul 28 14:24:31.173989 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:31.173909    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-fwf8p"
Jul 28 14:24:31.175553 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:31.175527    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dbm5c" event={"ID":"14c17f23-5dac-413e-9df4-bf19cfa8c4ef","Type":"ContainerStarted","Data":"eb9b77fdc12f9bb1d4ea2c0bda96aace3f6636f6c6c084eeebd7aa814478c398"}
Jul 28 14:24:31.175876 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:31.175850    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-dbm5c"
Jul 28 14:24:31.194937 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:31.194794    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt" podStartSLOduration=1.452551806 podStartE2EDuration="6.194778005s" totalImagesPullingTime="4.742226199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:31.193436967 +0000 UTC m=+340.994979180" watchObservedRunningTime="2026-07-28 14:24:31.194778005 +0000 UTC m=+340.996320219"
Jul 28 14:24:31.211867 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:31.211806    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-fwf8p" podStartSLOduration=1.760328457 podStartE2EDuration="7.21178833s" totalImagesPullingTime="5.451459873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:31.211297217 +0000 UTC m=+341.012839432" watchObservedRunningTime="2026-07-28 14:24:31.21178833 +0000 UTC m=+341.013330545"
Jul 28 14:24:31.229038 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:31.228981    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-dbm5c" podStartSLOduration=1.8383000470000002 podStartE2EDuration="6.228963983s" totalImagesPullingTime="4.390663936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:31.228036413 +0000 UTC m=+341.029578627" watchObservedRunningTime="2026-07-28 14:24:31.228963983 +0000 UTC m=+341.030506196"
Jul 28 14:24:32.183506 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.183471    2584 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-gr8wt container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 14:24:32.183884 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.183528    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt" podUID="c00eca47-39ef-4e49-9497-316dbee851dd" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 14:24:32.537076 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.536996    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"]
Jul 28 14:24:32.540328 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.540306    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.552514 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.552492    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-nxnmm\""
Jul 28 14:24:32.577174 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.577142    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"]
Jul 28 14:24:32.675013 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.674975    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a82f1155-e65d-46c1-aa7a-739022d05fc5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.675248 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.675026    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjr9l\" (UniqueName: \"kubernetes.io/projected/a82f1155-e65d-46c1-aa7a-739022d05fc5-kube-api-access-wjr9l\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.675248 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.675050    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a82f1155-e65d-46c1-aa7a-739022d05fc5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.675248 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.675149    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a82f1155-e65d-46c1-aa7a-739022d05fc5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.675248 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.675191    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a82f1155-e65d-46c1-aa7a-739022d05fc5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.675248 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.675239    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a82f1155-e65d-46c1-aa7a-739022d05fc5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.675492 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.675264    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a82f1155-e65d-46c1-aa7a-739022d05fc5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.675492 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.675314    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a82f1155-e65d-46c1-aa7a-739022d05fc5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.675492 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.675347    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a82f1155-e65d-46c1-aa7a-739022d05fc5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.775989 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.775943    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a82f1155-e65d-46c1-aa7a-739022d05fc5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.776189 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.776006    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a82f1155-e65d-46c1-aa7a-739022d05fc5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.776189 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.776033    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a82f1155-e65d-46c1-aa7a-739022d05fc5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.776189 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.776071    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a82f1155-e65d-46c1-aa7a-739022d05fc5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.776189 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.776109    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a82f1155-e65d-46c1-aa7a-739022d05fc5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.776401 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.776227    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a82f1155-e65d-46c1-aa7a-739022d05fc5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.776401 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.776260    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a82f1155-e65d-46c1-aa7a-739022d05fc5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.776401 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.776290    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a82f1155-e65d-46c1-aa7a-739022d05fc5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.776401 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.776347    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjr9l\" (UniqueName: \"kubernetes.io/projected/a82f1155-e65d-46c1-aa7a-739022d05fc5-kube-api-access-wjr9l\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.776627 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.776516    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a82f1155-e65d-46c1-aa7a-739022d05fc5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.776627 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.776594    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a82f1155-e65d-46c1-aa7a-739022d05fc5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.776799 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.776770    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a82f1155-e65d-46c1-aa7a-739022d05fc5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.776908 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.776881    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a82f1155-e65d-46c1-aa7a-739022d05fc5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.776908 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.776884    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a82f1155-e65d-46c1-aa7a-739022d05fc5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.778589 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.778568    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a82f1155-e65d-46c1-aa7a-739022d05fc5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.778790 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.778769    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a82f1155-e65d-46c1-aa7a-739022d05fc5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.792613 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.792536    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a82f1155-e65d-46c1-aa7a-739022d05fc5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.792743 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.792703    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjr9l\" (UniqueName: \"kubernetes.io/projected/a82f1155-e65d-46c1-aa7a-739022d05fc5-kube-api-access-wjr9l\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr\" (UID: \"a82f1155-e65d-46c1-aa7a-739022d05fc5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:32.855173 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:32.855132    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:33.004823 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:33.004794    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"]
Jul 28 14:24:33.006672 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:24:33.006643    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda82f1155_e65d_46c1_aa7a_739022d05fc5.slice/crio-fd94b723a6e7015b85f7b7422f76e95f492a3e58c3a1ca08b742b00c984f93a0 WatchSource:0}: Error finding container fd94b723a6e7015b85f7b7422f76e95f492a3e58c3a1ca08b742b00c984f93a0: Status 404 returned error can't find the container with id fd94b723a6e7015b85f7b7422f76e95f492a3e58c3a1ca08b742b00c984f93a0
Jul 28 14:24:33.186338 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:33.186302    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr" event={"ID":"a82f1155-e65d-46c1-aa7a-739022d05fc5","Type":"ContainerStarted","Data":"fd94b723a6e7015b85f7b7422f76e95f492a3e58c3a1ca08b742b00c984f93a0"}
Jul 28 14:24:35.183284 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:35.183243    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:35.773458 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:35.773424    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:24:35.773523 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:35.773497    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:24:35.773557 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:35.773526    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:24:35.949920 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:35.949883    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"]
Jul 28 14:24:35.953264 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:35.953249    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:35.979655 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:35.979627    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"]
Jul 28 14:24:36.105388 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.105357    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e8929d68-d91e-4163-bbfc-f848bba1f652-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.105558 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.105396    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8929d68-d91e-4163-bbfc-f848bba1f652-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.105558 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.105416    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8929d68-d91e-4163-bbfc-f848bba1f652-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.105558 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.105469    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66sjb\" (UniqueName: \"kubernetes.io/projected/e8929d68-d91e-4163-bbfc-f848bba1f652-kube-api-access-66sjb\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.105654 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.105563    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e8929d68-d91e-4163-bbfc-f848bba1f652-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.105654 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.105604    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8929d68-d91e-4163-bbfc-f848bba1f652-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.105654 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.105626    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e8929d68-d91e-4163-bbfc-f848bba1f652-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.201800 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.201758    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr" event={"ID":"a82f1155-e65d-46c1-aa7a-739022d05fc5","Type":"ContainerStarted","Data":"9f3ab975f110b403999190bf30643d026b31c6d55326afe06a7c3427c40d3d07"}
Jul 28 14:24:36.206433 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.206404    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e8929d68-d91e-4163-bbfc-f848bba1f652-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.206565 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.206446    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8929d68-d91e-4163-bbfc-f848bba1f652-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.206565 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.206472    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8929d68-d91e-4163-bbfc-f848bba1f652-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.206669 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.206645    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66sjb\" (UniqueName: \"kubernetes.io/projected/e8929d68-d91e-4163-bbfc-f848bba1f652-kube-api-access-66sjb\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.206732 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.206704    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e8929d68-d91e-4163-bbfc-f848bba1f652-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.206806 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.206783    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8929d68-d91e-4163-bbfc-f848bba1f652-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.206864 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.206826    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e8929d68-d91e-4163-bbfc-f848bba1f652-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.207494 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.207472    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e8929d68-d91e-4163-bbfc-f848bba1f652-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.208912 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.208892    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8929d68-d91e-4163-bbfc-f848bba1f652-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.209013 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.208925    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e8929d68-d91e-4163-bbfc-f848bba1f652-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.209241 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.209224    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8929d68-d91e-4163-bbfc-f848bba1f652-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.209334 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.209312    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e8929d68-d91e-4163-bbfc-f848bba1f652-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.220783 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.220754    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8929d68-d91e-4163-bbfc-f848bba1f652-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.258521 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.258480    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66sjb\" (UniqueName: \"kubernetes.io/projected/e8929d68-d91e-4163-bbfc-f848bba1f652-kube-api-access-66sjb\") pod \"istiod-openshift-gateway-5677ccb49b-4ndsh\" (UID: \"e8929d68-d91e-4163-bbfc-f848bba1f652\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.260899 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.260836    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr" podStartSLOduration=1.4962363349999999 podStartE2EDuration="4.260813672s" totalImagesPullingTime="2.764577337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:36.260517093 +0000 UTC m=+346.062059319" watchObservedRunningTime="2026-07-28 14:24:36.260813672 +0000 UTC m=+346.062355887"
Jul 28 14:24:36.264642 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.264621    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:36.424110 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.424070    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"]
Jul 28 14:24:36.425741 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:24:36.425714    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8929d68_d91e_4163_bbfc_f848bba1f652.slice/crio-1d21a91f513c1969e165b36e84d98400f317d07a31d97aa00ac489685e225fb5 WatchSource:0}: Error finding container 1d21a91f513c1969e165b36e84d98400f317d07a31d97aa00ac489685e225fb5: Status 404 returned error can't find the container with id 1d21a91f513c1969e165b36e84d98400f317d07a31d97aa00ac489685e225fb5
Jul 28 14:24:36.428701 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.428666    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:24:36.428789 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.428733    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:24:36.855371 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.855330    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:36.860592 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:36.860567    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:37.209649 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:37.209568    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh" event={"ID":"e8929d68-d91e-4163-bbfc-f848bba1f652","Type":"ContainerStarted","Data":"9f045da587fc8b9d03862ec6776e4f96ed354318e9090cf8355c4984a35aaf6c"}
Jul 28 14:24:37.209649 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:37.209622    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh" event={"ID":"e8929d68-d91e-4163-bbfc-f848bba1f652","Type":"ContainerStarted","Data":"1d21a91f513c1969e165b36e84d98400f317d07a31d97aa00ac489685e225fb5"}
Jul 28 14:24:37.210289 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:37.209846    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:37.210289 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:37.209881    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:37.210825 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:37.210801    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr"
Jul 28 14:24:37.211433 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:37.211398    2584 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-4ndsh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 14:24:37.211546 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:37.211463    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh" podUID="e8929d68-d91e-4163-bbfc-f848bba1f652" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 14:24:37.294837 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:37.294784    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh" podStartSLOduration=2.294760324 podStartE2EDuration="2.294760324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:37.291957685 +0000 UTC m=+347.093499899" watchObservedRunningTime="2026-07-28 14:24:37.294760324 +0000 UTC m=+347.096302540"
Jul 28 14:24:38.214380 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.214355    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ndsh"
Jul 28 14:24:38.352567 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.352539    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"]
Jul 28 14:24:38.352742 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.352697    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt" podUID="c00eca47-39ef-4e49-9497-316dbee851dd" containerName="discovery" containerID="cri-o://3aab80f8f4a0f7f0e88f88105d2cf007887cd5c509ce0c7b130990339b20d7d8" gracePeriod=30
Jul 28 14:24:38.607829 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.607800    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:38.730780 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.730701    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c00eca47-39ef-4e49-9497-316dbee851dd-istio-csr-ca-configmap\") pod \"c00eca47-39ef-4e49-9497-316dbee851dd\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") "
Jul 28 14:24:38.730780 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.730769    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c00eca47-39ef-4e49-9497-316dbee851dd-cacerts\") pod \"c00eca47-39ef-4e49-9497-316dbee851dd\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") "
Jul 28 14:24:38.730982 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.730804    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c00eca47-39ef-4e49-9497-316dbee851dd-istio-csr-dns-cert\") pod \"c00eca47-39ef-4e49-9497-316dbee851dd\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") "
Jul 28 14:24:38.730982 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.730841    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c00eca47-39ef-4e49-9497-316dbee851dd-istio-token\") pod \"c00eca47-39ef-4e49-9497-316dbee851dd\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") "
Jul 28 14:24:38.730982 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.730881    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gk9cg\" (UniqueName: \"kubernetes.io/projected/c00eca47-39ef-4e49-9497-316dbee851dd-kube-api-access-gk9cg\") pod \"c00eca47-39ef-4e49-9497-316dbee851dd\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") "
Jul 28 14:24:38.730982 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.730923    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c00eca47-39ef-4e49-9497-316dbee851dd-istio-kubeconfig\") pod \"c00eca47-39ef-4e49-9497-316dbee851dd\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") "
Jul 28 14:24:38.730982 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.730946    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c00eca47-39ef-4e49-9497-316dbee851dd-local-certs\") pod \"c00eca47-39ef-4e49-9497-316dbee851dd\" (UID: \"c00eca47-39ef-4e49-9497-316dbee851dd\") "
Jul 28 14:24:38.731257 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.731201    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c00eca47-39ef-4e49-9497-316dbee851dd-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "c00eca47-39ef-4e49-9497-316dbee851dd" (UID: "c00eca47-39ef-4e49-9497-316dbee851dd"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:24:38.733381 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.733348    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c00eca47-39ef-4e49-9497-316dbee851dd-kube-api-access-gk9cg" (OuterVolumeSpecName: "kube-api-access-gk9cg") pod "c00eca47-39ef-4e49-9497-316dbee851dd" (UID: "c00eca47-39ef-4e49-9497-316dbee851dd"). InnerVolumeSpecName "kube-api-access-gk9cg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:24:38.733506 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.733393    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c00eca47-39ef-4e49-9497-316dbee851dd-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "c00eca47-39ef-4e49-9497-316dbee851dd" (UID: "c00eca47-39ef-4e49-9497-316dbee851dd"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:24:38.733506 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.733397    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c00eca47-39ef-4e49-9497-316dbee851dd-istio-token" (OuterVolumeSpecName: "istio-token") pod "c00eca47-39ef-4e49-9497-316dbee851dd" (UID: "c00eca47-39ef-4e49-9497-316dbee851dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:24:38.733580 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.733529    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c00eca47-39ef-4e49-9497-316dbee851dd-cacerts" (OuterVolumeSpecName: "cacerts") pod "c00eca47-39ef-4e49-9497-316dbee851dd" (UID: "c00eca47-39ef-4e49-9497-316dbee851dd"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:24:38.733580 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.733526    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c00eca47-39ef-4e49-9497-316dbee851dd-local-certs" (OuterVolumeSpecName: "local-certs") pod "c00eca47-39ef-4e49-9497-316dbee851dd" (UID: "c00eca47-39ef-4e49-9497-316dbee851dd"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:24:38.733580 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.733551    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c00eca47-39ef-4e49-9497-316dbee851dd-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "c00eca47-39ef-4e49-9497-316dbee851dd" (UID: "c00eca47-39ef-4e49-9497-316dbee851dd"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:24:38.831970 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.831939    2584 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c00eca47-39ef-4e49-9497-316dbee851dd-cacerts\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:24:38.831970 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.831968    2584 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c00eca47-39ef-4e49-9497-316dbee851dd-istio-csr-dns-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:24:38.832193 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.831979    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c00eca47-39ef-4e49-9497-316dbee851dd-istio-token\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:24:38.832193 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.832010    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gk9cg\" (UniqueName: \"kubernetes.io/projected/c00eca47-39ef-4e49-9497-316dbee851dd-kube-api-access-gk9cg\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:24:38.832193 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.832020    2584 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c00eca47-39ef-4e49-9497-316dbee851dd-istio-kubeconfig\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:24:38.832193 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.832029    2584 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c00eca47-39ef-4e49-9497-316dbee851dd-local-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:24:38.832193 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:38.832036    2584 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c00eca47-39ef-4e49-9497-316dbee851dd-istio-csr-ca-configmap\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:24:39.219403 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:39.219370    2584 generic.go:356] "Generic (PLEG): container finished" podID="c00eca47-39ef-4e49-9497-316dbee851dd" containerID="3aab80f8f4a0f7f0e88f88105d2cf007887cd5c509ce0c7b130990339b20d7d8" exitCode=0
Jul 28 14:24:39.219839 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:39.219426    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"
Jul 28 14:24:39.219839 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:39.219448    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt" event={"ID":"c00eca47-39ef-4e49-9497-316dbee851dd","Type":"ContainerDied","Data":"3aab80f8f4a0f7f0e88f88105d2cf007887cd5c509ce0c7b130990339b20d7d8"}
Jul 28 14:24:39.219839 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:39.219540    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt" event={"ID":"c00eca47-39ef-4e49-9497-316dbee851dd","Type":"ContainerDied","Data":"d645f6ce5f0add53132e74eb7a0282ad8ad54d98fc4df31b652daeea000ce423"}
Jul 28 14:24:39.230608 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:39.219570    2584 scope.go:117] "RemoveContainer" containerID="3aab80f8f4a0f7f0e88f88105d2cf007887cd5c509ce0c7b130990339b20d7d8"
Jul 28 14:24:39.238521 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:39.238504    2584 scope.go:117] "RemoveContainer" containerID="3aab80f8f4a0f7f0e88f88105d2cf007887cd5c509ce0c7b130990339b20d7d8"
Jul 28 14:24:39.238817 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:24:39.238791    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3aab80f8f4a0f7f0e88f88105d2cf007887cd5c509ce0c7b130990339b20d7d8\": container with ID starting with 3aab80f8f4a0f7f0e88f88105d2cf007887cd5c509ce0c7b130990339b20d7d8 not found: ID does not exist" containerID="3aab80f8f4a0f7f0e88f88105d2cf007887cd5c509ce0c7b130990339b20d7d8"
Jul 28 14:24:39.238894 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:39.238826    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3aab80f8f4a0f7f0e88f88105d2cf007887cd5c509ce0c7b130990339b20d7d8"} err="failed to get container status \"3aab80f8f4a0f7f0e88f88105d2cf007887cd5c509ce0c7b130990339b20d7d8\": rpc error: code = NotFound desc = could not find container \"3aab80f8f4a0f7f0e88f88105d2cf007887cd5c509ce0c7b130990339b20d7d8\": container with ID starting with 3aab80f8f4a0f7f0e88f88105d2cf007887cd5c509ce0c7b130990339b20d7d8 not found: ID does not exist"
Jul 28 14:24:39.248293 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:39.248271    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"]
Jul 28 14:24:39.259880 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:39.259860    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gr8wt"]
Jul 28 14:24:40.796011 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:40.795974    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c00eca47-39ef-4e49-9497-316dbee851dd" path="/var/lib/kubelet/pods/c00eca47-39ef-4e49-9497-316dbee851dd/volumes"
Jul 28 14:24:42.183926 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:42.183841    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-fwf8p"
Jul 28 14:24:42.186248 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:24:42.186214    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-dbm5c"
Jul 28 14:25:25.427063 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:25.427008    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gm82m"]
Jul 28 14:25:25.427548 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:25.427311    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c00eca47-39ef-4e49-9497-316dbee851dd" containerName="discovery"
Jul 28 14:25:25.427548 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:25.427323    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c00eca47-39ef-4e49-9497-316dbee851dd" containerName="discovery"
Jul 28 14:25:25.427548 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:25.427381    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c00eca47-39ef-4e49-9497-316dbee851dd" containerName="discovery"
Jul 28 14:25:25.430169 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:25.430154    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gm82m"
Jul 28 14:25:25.434481 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:25.434455    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 14:25:25.434649 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:25.434534    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-f5lnh\""
Jul 28 14:25:25.434948 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:25.434926    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 14:25:25.449680 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:25.449649    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gm82m"]
Jul 28 14:25:25.537530 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:25.537494    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwssk\" (UniqueName: \"kubernetes.io/projected/657054fd-6213-4abb-b99d-9a38fb48f660-kube-api-access-hwssk\") pod \"kuadrant-operator-catalog-gm82m\" (UID: \"657054fd-6213-4abb-b99d-9a38fb48f660\") " pod="kuadrant-system/kuadrant-operator-catalog-gm82m"
Jul 28 14:25:25.638050 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:25.638009    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwssk\" (UniqueName: \"kubernetes.io/projected/657054fd-6213-4abb-b99d-9a38fb48f660-kube-api-access-hwssk\") pod \"kuadrant-operator-catalog-gm82m\" (UID: \"657054fd-6213-4abb-b99d-9a38fb48f660\") " pod="kuadrant-system/kuadrant-operator-catalog-gm82m"
Jul 28 14:25:25.649461 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:25.649426    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwssk\" (UniqueName: \"kubernetes.io/projected/657054fd-6213-4abb-b99d-9a38fb48f660-kube-api-access-hwssk\") pod \"kuadrant-operator-catalog-gm82m\" (UID: \"657054fd-6213-4abb-b99d-9a38fb48f660\") " pod="kuadrant-system/kuadrant-operator-catalog-gm82m"
Jul 28 14:25:25.740251 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:25.740159    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gm82m"
Jul 28 14:25:25.757281 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:25.757255    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gm82m"]
Jul 28 14:25:25.880124 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:25.880083    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gm82m"]
Jul 28 14:25:25.882713 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:25:25.882686    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod657054fd_6213_4abb_b99d_9a38fb48f660.slice/crio-90c5f630bc396b4d4e953db9f6167bed5a93e4cad90da2f0f3d85a414f00ee0d WatchSource:0}: Error finding container 90c5f630bc396b4d4e953db9f6167bed5a93e4cad90da2f0f3d85a414f00ee0d: Status 404 returned error can't find the container with id 90c5f630bc396b4d4e953db9f6167bed5a93e4cad90da2f0f3d85a414f00ee0d
Jul 28 14:25:25.969872 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:25.969841    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hgs74"]
Jul 28 14:25:25.974141 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:25.974119    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hgs74"
Jul 28 14:25:25.988681 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:25.988655    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hgs74"]
Jul 28 14:25:26.043298 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:26.043199    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6fsv\" (UniqueName: \"kubernetes.io/projected/7922ad58-46bf-4c59-9713-162ae3ec1bc7-kube-api-access-t6fsv\") pod \"kuadrant-operator-catalog-hgs74\" (UID: \"7922ad58-46bf-4c59-9713-162ae3ec1bc7\") " pod="kuadrant-system/kuadrant-operator-catalog-hgs74"
Jul 28 14:25:26.143617 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:26.143576    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6fsv\" (UniqueName: \"kubernetes.io/projected/7922ad58-46bf-4c59-9713-162ae3ec1bc7-kube-api-access-t6fsv\") pod \"kuadrant-operator-catalog-hgs74\" (UID: \"7922ad58-46bf-4c59-9713-162ae3ec1bc7\") " pod="kuadrant-system/kuadrant-operator-catalog-hgs74"
Jul 28 14:25:26.163759 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:26.163726    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6fsv\" (UniqueName: \"kubernetes.io/projected/7922ad58-46bf-4c59-9713-162ae3ec1bc7-kube-api-access-t6fsv\") pod \"kuadrant-operator-catalog-hgs74\" (UID: \"7922ad58-46bf-4c59-9713-162ae3ec1bc7\") " pod="kuadrant-system/kuadrant-operator-catalog-hgs74"
Jul 28 14:25:26.283550 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:26.283501    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hgs74"
Jul 28 14:25:26.411263 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:26.411232    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gm82m" event={"ID":"657054fd-6213-4abb-b99d-9a38fb48f660","Type":"ContainerStarted","Data":"90c5f630bc396b4d4e953db9f6167bed5a93e4cad90da2f0f3d85a414f00ee0d"}
Jul 28 14:25:26.420628 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:26.420600    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hgs74"]
Jul 28 14:25:26.423422 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:25:26.423395    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7922ad58_46bf_4c59_9713_162ae3ec1bc7.slice/crio-2f19d215266455a70cc27d20168c7208fab1495ba90d1a057cdb853a638ea917 WatchSource:0}: Error finding container 2f19d215266455a70cc27d20168c7208fab1495ba90d1a057cdb853a638ea917: Status 404 returned error can't find the container with id 2f19d215266455a70cc27d20168c7208fab1495ba90d1a057cdb853a638ea917
Jul 28 14:25:27.416387 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:27.416354    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hgs74" event={"ID":"7922ad58-46bf-4c59-9713-162ae3ec1bc7","Type":"ContainerStarted","Data":"2f19d215266455a70cc27d20168c7208fab1495ba90d1a057cdb853a638ea917"}
Jul 28 14:25:29.427042 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:29.426939    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hgs74" event={"ID":"7922ad58-46bf-4c59-9713-162ae3ec1bc7","Type":"ContainerStarted","Data":"eaee29e24c9e3b5c10a6f77fd3d734a73598d97580f089dfbec485052607bc3f"}
Jul 28 14:25:29.428386 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:29.428362    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gm82m" event={"ID":"657054fd-6213-4abb-b99d-9a38fb48f660","Type":"ContainerStarted","Data":"72cc313db800d6aa9d2e59384011712bc52e09454c8bef6239bc859e46c7e5df"}
Jul 28 14:25:29.428504 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:29.428441    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-gm82m" podUID="657054fd-6213-4abb-b99d-9a38fb48f660" containerName="registry-server" containerID="cri-o://72cc313db800d6aa9d2e59384011712bc52e09454c8bef6239bc859e46c7e5df" gracePeriod=2
Jul 28 14:25:29.445765 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:29.445720    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-hgs74" podStartSLOduration=1.739624622 podStartE2EDuration="4.445704036s" totalImagesPullingTime="2.706079414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:25:29.444362222 +0000 UTC m=+399.245904449" watchObservedRunningTime="2026-07-28 14:25:29.445704036 +0000 UTC m=+399.247246248"
Jul 28 14:25:29.466739 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:29.466679    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-gm82m" podStartSLOduration=1.217493376 podStartE2EDuration="4.466659622s" totalImagesPullingTime="3.249166246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:25:29.466049231 +0000 UTC m=+399.267591460" watchObservedRunningTime="2026-07-28 14:25:29.466659622 +0000 UTC m=+399.268201839"
Jul 28 14:25:29.667914 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:29.667888    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gm82m"
Jul 28 14:25:29.775744 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:29.775649    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwssk\" (UniqueName: \"kubernetes.io/projected/657054fd-6213-4abb-b99d-9a38fb48f660-kube-api-access-hwssk\") pod \"657054fd-6213-4abb-b99d-9a38fb48f660\" (UID: \"657054fd-6213-4abb-b99d-9a38fb48f660\") "
Jul 28 14:25:29.777822 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:29.777795    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/657054fd-6213-4abb-b99d-9a38fb48f660-kube-api-access-hwssk" (OuterVolumeSpecName: "kube-api-access-hwssk") pod "657054fd-6213-4abb-b99d-9a38fb48f660" (UID: "657054fd-6213-4abb-b99d-9a38fb48f660"). InnerVolumeSpecName "kube-api-access-hwssk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:25:29.877009 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:29.876978    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwssk\" (UniqueName: \"kubernetes.io/projected/657054fd-6213-4abb-b99d-9a38fb48f660-kube-api-access-hwssk\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:25:30.433538 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:30.433453    2584 generic.go:356] "Generic (PLEG): container finished" podID="657054fd-6213-4abb-b99d-9a38fb48f660" containerID="72cc313db800d6aa9d2e59384011712bc52e09454c8bef6239bc859e46c7e5df" exitCode=0
Jul 28 14:25:30.433538 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:30.433498    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gm82m"
Jul 28 14:25:30.434017 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:30.433536    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gm82m" event={"ID":"657054fd-6213-4abb-b99d-9a38fb48f660","Type":"ContainerDied","Data":"72cc313db800d6aa9d2e59384011712bc52e09454c8bef6239bc859e46c7e5df"}
Jul 28 14:25:30.434017 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:30.433584    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gm82m" event={"ID":"657054fd-6213-4abb-b99d-9a38fb48f660","Type":"ContainerDied","Data":"90c5f630bc396b4d4e953db9f6167bed5a93e4cad90da2f0f3d85a414f00ee0d"}
Jul 28 14:25:30.434017 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:30.433604    2584 scope.go:117] "RemoveContainer" containerID="72cc313db800d6aa9d2e59384011712bc52e09454c8bef6239bc859e46c7e5df"
Jul 28 14:25:30.442479 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:30.442465    2584 scope.go:117] "RemoveContainer" containerID="72cc313db800d6aa9d2e59384011712bc52e09454c8bef6239bc859e46c7e5df"
Jul 28 14:25:30.442715 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:25:30.442697    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72cc313db800d6aa9d2e59384011712bc52e09454c8bef6239bc859e46c7e5df\": container with ID starting with 72cc313db800d6aa9d2e59384011712bc52e09454c8bef6239bc859e46c7e5df not found: ID does not exist" containerID="72cc313db800d6aa9d2e59384011712bc52e09454c8bef6239bc859e46c7e5df"
Jul 28 14:25:30.442762 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:30.442722    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72cc313db800d6aa9d2e59384011712bc52e09454c8bef6239bc859e46c7e5df"} err="failed to get container status \"72cc313db800d6aa9d2e59384011712bc52e09454c8bef6239bc859e46c7e5df\": rpc error: code = NotFound desc = could not find container \"72cc313db800d6aa9d2e59384011712bc52e09454c8bef6239bc859e46c7e5df\": container with ID starting with 72cc313db800d6aa9d2e59384011712bc52e09454c8bef6239bc859e46c7e5df not found: ID does not exist"
Jul 28 14:25:30.456296 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:30.456274    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gm82m"]
Jul 28 14:25:30.461479 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:30.461456    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gm82m"]
Jul 28 14:25:30.795109 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:30.795015    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="657054fd-6213-4abb-b99d-9a38fb48f660" path="/var/lib/kubelet/pods/657054fd-6213-4abb-b99d-9a38fb48f660/volumes"
Jul 28 14:25:36.283898 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:36.283862    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-hgs74"
Jul 28 14:25:36.284394 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:36.283945    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-hgs74"
Jul 28 14:25:36.304911 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:36.304875    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-hgs74"
Jul 28 14:25:36.479615 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:36.479588    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-hgs74"
Jul 28 14:25:57.538339 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:57.538304    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht"]
Jul 28 14:25:57.538863 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:57.538638    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="657054fd-6213-4abb-b99d-9a38fb48f660" containerName="registry-server"
Jul 28 14:25:57.538863 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:57.538654    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="657054fd-6213-4abb-b99d-9a38fb48f660" containerName="registry-server"
Jul 28 14:25:57.538863 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:57.538731    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="657054fd-6213-4abb-b99d-9a38fb48f660" containerName="registry-server"
Jul 28 14:25:57.542939 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:57.542918    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht"
Jul 28 14:25:57.545662 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:57.545645    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-p57sk\""
Jul 28 14:25:57.564767 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:57.564740    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht"]
Jul 28 14:25:57.595912 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:57.595876    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjnnp\" (UniqueName: \"kubernetes.io/projected/413104ba-ae1d-4cc5-b2ef-e4475a6a8f18-kube-api-access-wjnnp\") pod \"limitador-operator-controller-manager-859d7b78bd-ss8ht\" (UID: \"413104ba-ae1d-4cc5-b2ef-e4475a6a8f18\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht"
Jul 28 14:25:57.697146 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:57.697074    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjnnp\" (UniqueName: \"kubernetes.io/projected/413104ba-ae1d-4cc5-b2ef-e4475a6a8f18-kube-api-access-wjnnp\") pod \"limitador-operator-controller-manager-859d7b78bd-ss8ht\" (UID: \"413104ba-ae1d-4cc5-b2ef-e4475a6a8f18\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht"
Jul 28 14:25:57.707303 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:57.707273    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjnnp\" (UniqueName: \"kubernetes.io/projected/413104ba-ae1d-4cc5-b2ef-e4475a6a8f18-kube-api-access-wjnnp\") pod \"limitador-operator-controller-manager-859d7b78bd-ss8ht\" (UID: \"413104ba-ae1d-4cc5-b2ef-e4475a6a8f18\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht"
Jul 28 14:25:57.852735 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:57.852701    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht"
Jul 28 14:25:57.984752 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:57.984717    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht"]
Jul 28 14:25:57.988725 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:25:57.988700    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod413104ba_ae1d_4cc5_b2ef_e4475a6a8f18.slice/crio-7384811c3996dc8b940930d6d9ac27115b4cc5337389738affaab2a065a75264 WatchSource:0}: Error finding container 7384811c3996dc8b940930d6d9ac27115b4cc5337389738affaab2a065a75264: Status 404 returned error can't find the container with id 7384811c3996dc8b940930d6d9ac27115b4cc5337389738affaab2a065a75264
Jul 28 14:25:58.547488 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:58.547448    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht" event={"ID":"413104ba-ae1d-4cc5-b2ef-e4475a6a8f18","Type":"ContainerStarted","Data":"7384811c3996dc8b940930d6d9ac27115b4cc5337389738affaab2a065a75264"}
Jul 28 14:25:59.553195 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:59.553160    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht" event={"ID":"413104ba-ae1d-4cc5-b2ef-e4475a6a8f18","Type":"ContainerStarted","Data":"d297b8d000a480c2713852ef766fb1e08889b1e645753ca9bbcb091058ea27b6"}
Jul 28 14:25:59.553568 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:59.553307    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht"
Jul 28 14:25:59.574377 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:25:59.574303    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht" podStartSLOduration=1.105533688 podStartE2EDuration="2.574282765s" totalImagesPullingTime="1.468749077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:25:59.573040862 +0000 UTC m=+429.374583075" watchObservedRunningTime="2026-07-28 14:25:59.574282765 +0000 UTC m=+429.375824980"
Jul 28 14:26:10.559414 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:10.559384    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht"
Jul 28 14:26:28.062034 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.061952    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht"]
Jul 28 14:26:28.062497 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.062197    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht" podUID="413104ba-ae1d-4cc5-b2ef-e4475a6a8f18" containerName="manager" containerID="cri-o://d297b8d000a480c2713852ef766fb1e08889b1e645753ca9bbcb091058ea27b6" gracePeriod=2
Jul 28 14:26:28.083427 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.083398    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht"]
Jul 28 14:26:28.099143 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.099086    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j"]
Jul 28 14:26:28.099557 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.099543    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="413104ba-ae1d-4cc5-b2ef-e4475a6a8f18" containerName="manager"
Jul 28 14:26:28.099603 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.099561    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="413104ba-ae1d-4cc5-b2ef-e4475a6a8f18" containerName="manager"
Jul 28 14:26:28.099663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.099652    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="413104ba-ae1d-4cc5-b2ef-e4475a6a8f18" containerName="manager"
Jul 28 14:26:28.101903 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.101879    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j"
Jul 28 14:26:28.115157 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:26:28.115119    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ss8ht\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="413104ba-ae1d-4cc5-b2ef-e4475a6a8f18" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht"
Jul 28 14:26:28.115337 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.115231    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dnzzf\""
Jul 28 14:26:28.132971 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.132943    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j"]
Jul 28 14:26:28.257456 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.257431    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dae385c4-6d25-47ff-98a5-70e6178ca6d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wpn9j\" (UID: \"dae385c4-6d25-47ff-98a5-70e6178ca6d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j"
Jul 28 14:26:28.257583 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.257496    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxhnj\" (UniqueName: \"kubernetes.io/projected/dae385c4-6d25-47ff-98a5-70e6178ca6d8-kube-api-access-lxhnj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wpn9j\" (UID: \"dae385c4-6d25-47ff-98a5-70e6178ca6d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j"
Jul 28 14:26:28.318837 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.318773    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht"
Jul 28 14:26:28.321475 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:26:28.321447    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ss8ht\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="413104ba-ae1d-4cc5-b2ef-e4475a6a8f18" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht"
Jul 28 14:26:28.358511 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.358477    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dae385c4-6d25-47ff-98a5-70e6178ca6d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wpn9j\" (UID: \"dae385c4-6d25-47ff-98a5-70e6178ca6d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j"
Jul 28 14:26:28.358511 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.358528    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxhnj\" (UniqueName: \"kubernetes.io/projected/dae385c4-6d25-47ff-98a5-70e6178ca6d8-kube-api-access-lxhnj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wpn9j\" (UID: \"dae385c4-6d25-47ff-98a5-70e6178ca6d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j"
Jul 28 14:26:28.358875 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.358857    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dae385c4-6d25-47ff-98a5-70e6178ca6d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wpn9j\" (UID: \"dae385c4-6d25-47ff-98a5-70e6178ca6d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j"
Jul 28 14:26:28.381445 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.381415    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxhnj\" (UniqueName: \"kubernetes.io/projected/dae385c4-6d25-47ff-98a5-70e6178ca6d8-kube-api-access-lxhnj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wpn9j\" (UID: \"dae385c4-6d25-47ff-98a5-70e6178ca6d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j"
Jul 28 14:26:28.445418 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.445381    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j"
Jul 28 14:26:28.459575 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.459546    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjnnp\" (UniqueName: \"kubernetes.io/projected/413104ba-ae1d-4cc5-b2ef-e4475a6a8f18-kube-api-access-wjnnp\") pod \"413104ba-ae1d-4cc5-b2ef-e4475a6a8f18\" (UID: \"413104ba-ae1d-4cc5-b2ef-e4475a6a8f18\") "
Jul 28 14:26:28.461502 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.461473    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/413104ba-ae1d-4cc5-b2ef-e4475a6a8f18-kube-api-access-wjnnp" (OuterVolumeSpecName: "kube-api-access-wjnnp") pod "413104ba-ae1d-4cc5-b2ef-e4475a6a8f18" (UID: "413104ba-ae1d-4cc5-b2ef-e4475a6a8f18"). InnerVolumeSpecName "kube-api-access-wjnnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:26:28.560902 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.560865    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjnnp\" (UniqueName: \"kubernetes.io/projected/413104ba-ae1d-4cc5-b2ef-e4475a6a8f18-kube-api-access-wjnnp\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:26:28.619398 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:26:28.619364    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddae385c4_6d25_47ff_98a5_70e6178ca6d8.slice/crio-157fd63112931c43eb1937531dc0f1d6b5fa8de9bcb5fafb141974ae06a43713 WatchSource:0}: Error finding container 157fd63112931c43eb1937531dc0f1d6b5fa8de9bcb5fafb141974ae06a43713: Status 404 returned error can't find the container with id 157fd63112931c43eb1937531dc0f1d6b5fa8de9bcb5fafb141974ae06a43713
Jul 28 14:26:28.624824 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.624798    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j"]
Jul 28 14:26:28.670570 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.670532    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j" event={"ID":"dae385c4-6d25-47ff-98a5-70e6178ca6d8","Type":"ContainerStarted","Data":"157fd63112931c43eb1937531dc0f1d6b5fa8de9bcb5fafb141974ae06a43713"}
Jul 28 14:26:28.671825 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.671795    2584 generic.go:356] "Generic (PLEG): container finished" podID="413104ba-ae1d-4cc5-b2ef-e4475a6a8f18" containerID="d297b8d000a480c2713852ef766fb1e08889b1e645753ca9bbcb091058ea27b6" exitCode=0
Jul 28 14:26:28.671945 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.671840    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht"
Jul 28 14:26:28.671945 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.671870    2584 scope.go:117] "RemoveContainer" containerID="d297b8d000a480c2713852ef766fb1e08889b1e645753ca9bbcb091058ea27b6"
Jul 28 14:26:28.675069 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:26:28.675046    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ss8ht\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="413104ba-ae1d-4cc5-b2ef-e4475a6a8f18" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht"
Jul 28 14:26:28.681034 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.681008    2584 scope.go:117] "RemoveContainer" containerID="d297b8d000a480c2713852ef766fb1e08889b1e645753ca9bbcb091058ea27b6"
Jul 28 14:26:28.681361 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:26:28.681341    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d297b8d000a480c2713852ef766fb1e08889b1e645753ca9bbcb091058ea27b6\": container with ID starting with d297b8d000a480c2713852ef766fb1e08889b1e645753ca9bbcb091058ea27b6 not found: ID does not exist" containerID="d297b8d000a480c2713852ef766fb1e08889b1e645753ca9bbcb091058ea27b6"
Jul 28 14:26:28.681421 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.681371    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d297b8d000a480c2713852ef766fb1e08889b1e645753ca9bbcb091058ea27b6"} err="failed to get container status \"d297b8d000a480c2713852ef766fb1e08889b1e645753ca9bbcb091058ea27b6\": rpc error: code = NotFound desc = could not find container \"d297b8d000a480c2713852ef766fb1e08889b1e645753ca9bbcb091058ea27b6\": container with ID starting with d297b8d000a480c2713852ef766fb1e08889b1e645753ca9bbcb091058ea27b6 not found: ID does not exist"
Jul 28 14:26:28.683275 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:26:28.683254    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ss8ht\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="413104ba-ae1d-4cc5-b2ef-e4475a6a8f18" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ss8ht"
Jul 28 14:26:28.796352 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:28.796317    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="413104ba-ae1d-4cc5-b2ef-e4475a6a8f18" path="/var/lib/kubelet/pods/413104ba-ae1d-4cc5-b2ef-e4475a6a8f18/volumes"
Jul 28 14:26:34.700014 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:34.699979    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j" event={"ID":"dae385c4-6d25-47ff-98a5-70e6178ca6d8","Type":"ContainerStarted","Data":"36a16d201929d6cafc7392d91ecc873cdf51d9710314ac511248ba81d382861b"}
Jul 28 14:26:34.700437 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:34.700201    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j"
Jul 28 14:26:34.747292 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:34.747233    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j" podStartSLOduration=1.306787449 podStartE2EDuration="6.747217525s" totalImagesPullingTime="5.440430076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:26:34.74329756 +0000 UTC m=+464.544839777" watchObservedRunningTime="2026-07-28 14:26:34.747217525 +0000 UTC m=+464.548759741"
Jul 28 14:26:43.569421 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:43.569381    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j"]
Jul 28 14:26:43.569913 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:43.569647    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j" podUID="dae385c4-6d25-47ff-98a5-70e6178ca6d8" containerName="manager" containerID="cri-o://36a16d201929d6cafc7392d91ecc873cdf51d9710314ac511248ba81d382861b" gracePeriod=10
Jul 28 14:26:43.571560 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:43.571539    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j"
Jul 28 14:26:43.740372 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:43.740342    2584 generic.go:356] "Generic (PLEG): container finished" podID="dae385c4-6d25-47ff-98a5-70e6178ca6d8" containerID="36a16d201929d6cafc7392d91ecc873cdf51d9710314ac511248ba81d382861b" exitCode=0
Jul 28 14:26:43.740538 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:43.740416    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j" event={"ID":"dae385c4-6d25-47ff-98a5-70e6178ca6d8","Type":"ContainerDied","Data":"36a16d201929d6cafc7392d91ecc873cdf51d9710314ac511248ba81d382861b"}
Jul 28 14:26:43.828161 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:43.828082    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j"
Jul 28 14:26:43.885299 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:43.885264    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dae385c4-6d25-47ff-98a5-70e6178ca6d8-extensions-socket-volume\") pod \"dae385c4-6d25-47ff-98a5-70e6178ca6d8\" (UID: \"dae385c4-6d25-47ff-98a5-70e6178ca6d8\") "
Jul 28 14:26:43.885498 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:43.885366    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxhnj\" (UniqueName: \"kubernetes.io/projected/dae385c4-6d25-47ff-98a5-70e6178ca6d8-kube-api-access-lxhnj\") pod \"dae385c4-6d25-47ff-98a5-70e6178ca6d8\" (UID: \"dae385c4-6d25-47ff-98a5-70e6178ca6d8\") "
Jul 28 14:26:43.885741 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:43.885718    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dae385c4-6d25-47ff-98a5-70e6178ca6d8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dae385c4-6d25-47ff-98a5-70e6178ca6d8" (UID: "dae385c4-6d25-47ff-98a5-70e6178ca6d8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:26:43.887477 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:43.887451    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dae385c4-6d25-47ff-98a5-70e6178ca6d8-kube-api-access-lxhnj" (OuterVolumeSpecName: "kube-api-access-lxhnj") pod "dae385c4-6d25-47ff-98a5-70e6178ca6d8" (UID: "dae385c4-6d25-47ff-98a5-70e6178ca6d8"). InnerVolumeSpecName "kube-api-access-lxhnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:26:43.986657 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:43.986616    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxhnj\" (UniqueName: \"kubernetes.io/projected/dae385c4-6d25-47ff-98a5-70e6178ca6d8-kube-api-access-lxhnj\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:26:43.986657 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:43.986650    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dae385c4-6d25-47ff-98a5-70e6178ca6d8-extensions-socket-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:26:44.746691 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:44.746654    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j" event={"ID":"dae385c4-6d25-47ff-98a5-70e6178ca6d8","Type":"ContainerDied","Data":"157fd63112931c43eb1937531dc0f1d6b5fa8de9bcb5fafb141974ae06a43713"}
Jul 28 14:26:44.746691 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:44.746689    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j"
Jul 28 14:26:44.747230 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:44.746729    2584 scope.go:117] "RemoveContainer" containerID="36a16d201929d6cafc7392d91ecc873cdf51d9710314ac511248ba81d382861b"
Jul 28 14:26:44.774299 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:44.774263    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j"]
Jul 28 14:26:44.780694 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:44.780660    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wpn9j"]
Jul 28 14:26:44.795908 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:26:44.795881    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dae385c4-6d25-47ff-98a5-70e6178ca6d8" path="/var/lib/kubelet/pods/dae385c4-6d25-47ff-98a5-70e6178ca6d8/volumes"
Jul 28 14:27:00.069964 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.069928    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"]
Jul 28 14:27:00.070527 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.070273    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dae385c4-6d25-47ff-98a5-70e6178ca6d8" containerName="manager"
Jul 28 14:27:00.070527 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.070285    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="dae385c4-6d25-47ff-98a5-70e6178ca6d8" containerName="manager"
Jul 28 14:27:00.070527 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.070338    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="dae385c4-6d25-47ff-98a5-70e6178ca6d8" containerName="manager"
Jul 28 14:27:00.075159 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.075132    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.078111 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.078074    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-q725k\""
Jul 28 14:27:00.091371 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.091336    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"]
Jul 28 14:27:00.115753 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.115711    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0341cb69-6aaa-47f2-8009-28908c0830c7-istio-envoy\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.115973 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.115777    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0341cb69-6aaa-47f2-8009-28908c0830c7-credential-socket\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.115973 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.115804    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0341cb69-6aaa-47f2-8009-28908c0830c7-workload-socket\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.115973 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.115826    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0341cb69-6aaa-47f2-8009-28908c0830c7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.115973 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.115856    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0341cb69-6aaa-47f2-8009-28908c0830c7-workload-certs\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.115973 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.115885    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0341cb69-6aaa-47f2-8009-28908c0830c7-istio-data\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.115973 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.115910    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0341cb69-6aaa-47f2-8009-28908c0830c7-istio-token\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.116233 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.115988    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0341cb69-6aaa-47f2-8009-28908c0830c7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.116233 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.116056    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rxjz\" (UniqueName: \"kubernetes.io/projected/0341cb69-6aaa-47f2-8009-28908c0830c7-kube-api-access-2rxjz\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.216629 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.216580    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0341cb69-6aaa-47f2-8009-28908c0830c7-istio-envoy\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.216838 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.216641    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0341cb69-6aaa-47f2-8009-28908c0830c7-credential-socket\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.216838 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.216666    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0341cb69-6aaa-47f2-8009-28908c0830c7-workload-socket\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.216838 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.216688    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0341cb69-6aaa-47f2-8009-28908c0830c7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.216838 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.216718    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0341cb69-6aaa-47f2-8009-28908c0830c7-workload-certs\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.216838 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.216753    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0341cb69-6aaa-47f2-8009-28908c0830c7-istio-data\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.217121 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.216855    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0341cb69-6aaa-47f2-8009-28908c0830c7-istio-token\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.217121 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.216907    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0341cb69-6aaa-47f2-8009-28908c0830c7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.217121 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.216946    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rxjz\" (UniqueName: \"kubernetes.io/projected/0341cb69-6aaa-47f2-8009-28908c0830c7-kube-api-access-2rxjz\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.217279 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.217156    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0341cb69-6aaa-47f2-8009-28908c0830c7-workload-certs\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.217279 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.217230    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0341cb69-6aaa-47f2-8009-28908c0830c7-workload-socket\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.217384 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.217290    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0341cb69-6aaa-47f2-8009-28908c0830c7-credential-socket\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.217384 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.217327    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0341cb69-6aaa-47f2-8009-28908c0830c7-istio-data\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.217548 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.217517    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0341cb69-6aaa-47f2-8009-28908c0830c7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.219002 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.218981    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0341cb69-6aaa-47f2-8009-28908c0830c7-istio-envoy\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.219486 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.219468    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0341cb69-6aaa-47f2-8009-28908c0830c7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.231974 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.231947    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0341cb69-6aaa-47f2-8009-28908c0830c7-istio-token\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.232169 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.232148    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rxjz\" (UniqueName: \"kubernetes.io/projected/0341cb69-6aaa-47f2-8009-28908c0830c7-kube-api-access-2rxjz\") pod \"maas-default-gateway-openshift-default-c747fcbf8-4nzkp\" (UID: \"0341cb69-6aaa-47f2-8009-28908c0830c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.385514 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.385470    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:00.531183 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.531056    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"]
Jul 28 14:27:00.533721 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:27:00.533695    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0341cb69_6aaa_47f2_8009_28908c0830c7.slice/crio-f7dd8125f7b22bd1931f4a27c32da445009529ca58c3a9fb51226687b2474d90 WatchSource:0}: Error finding container f7dd8125f7b22bd1931f4a27c32da445009529ca58c3a9fb51226687b2474d90: Status 404 returned error can't find the container with id f7dd8125f7b22bd1931f4a27c32da445009529ca58c3a9fb51226687b2474d90
Jul 28 14:27:00.535706 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.535663    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:27:00.535779 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.535745    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:27:00.535779 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.535773    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:27:00.813732 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.813628    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp" event={"ID":"0341cb69-6aaa-47f2-8009-28908c0830c7","Type":"ContainerStarted","Data":"2a835537a483b022daa1a51e50f6f482bde0f677ea1b51d080abe00fe0897e1d"}
Jul 28 14:27:00.813732 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.813695    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp" event={"ID":"0341cb69-6aaa-47f2-8009-28908c0830c7","Type":"ContainerStarted","Data":"f7dd8125f7b22bd1931f4a27c32da445009529ca58c3a9fb51226687b2474d90"}
Jul 28 14:27:00.839582 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:00.839529    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp" podStartSLOduration=0.839510089 podStartE2EDuration="839.510089ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:27:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:27:00.838075501 +0000 UTC m=+490.639617717" watchObservedRunningTime="2026-07-28 14:27:00.839510089 +0000 UTC m=+490.641052332"
Jul 28 14:27:01.386039 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:01.385992    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:01.390699 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:01.390672    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:01.818020 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:01.817940    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:01.819019 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:01.819003    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:27:01.955435 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:01.955394    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bmps4"]
Jul 28 14:27:01.958630 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:01.958612    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bmps4"
Jul 28 14:27:01.962158 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:01.962133    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 14:27:01.962309 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:01.962133    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-28hpc\""
Jul 28 14:27:01.973116 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:01.973067    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bmps4"]
Jul 28 14:27:02.032813 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:02.032777    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2be2218c-f24d-4029-8b68-20b28b22767e-config-file\") pod \"limitador-limitador-64c94fd64f-bmps4\" (UID: \"2be2218c-f24d-4029-8b68-20b28b22767e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bmps4"
Jul 28 14:27:02.032979 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:02.032858    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94zpk\" (UniqueName: \"kubernetes.io/projected/2be2218c-f24d-4029-8b68-20b28b22767e-kube-api-access-94zpk\") pod \"limitador-limitador-64c94fd64f-bmps4\" (UID: \"2be2218c-f24d-4029-8b68-20b28b22767e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bmps4"
Jul 28 14:27:02.051374 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:02.051337    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bmps4"]
Jul 28 14:27:02.134233 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:02.134195    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2be2218c-f24d-4029-8b68-20b28b22767e-config-file\") pod \"limitador-limitador-64c94fd64f-bmps4\" (UID: \"2be2218c-f24d-4029-8b68-20b28b22767e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bmps4"
Jul 28 14:27:02.134432 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:02.134262    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94zpk\" (UniqueName: \"kubernetes.io/projected/2be2218c-f24d-4029-8b68-20b28b22767e-kube-api-access-94zpk\") pod \"limitador-limitador-64c94fd64f-bmps4\" (UID: \"2be2218c-f24d-4029-8b68-20b28b22767e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bmps4"
Jul 28 14:27:02.134940 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:02.134920    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2be2218c-f24d-4029-8b68-20b28b22767e-config-file\") pod \"limitador-limitador-64c94fd64f-bmps4\" (UID: \"2be2218c-f24d-4029-8b68-20b28b22767e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bmps4"
Jul 28 14:27:02.146279 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:02.146240    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94zpk\" (UniqueName: \"kubernetes.io/projected/2be2218c-f24d-4029-8b68-20b28b22767e-kube-api-access-94zpk\") pod \"limitador-limitador-64c94fd64f-bmps4\" (UID: \"2be2218c-f24d-4029-8b68-20b28b22767e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bmps4"
Jul 28 14:27:02.269264 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:02.269227    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bmps4"
Jul 28 14:27:02.408791 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:02.408767    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bmps4"]
Jul 28 14:27:02.411333 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:27:02.411304    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2be2218c_f24d_4029_8b68_20b28b22767e.slice/crio-d81bbd1e64a3f31434241fbc652f30f0303c7dcf7902c32319def4cf09cc235b WatchSource:0}: Error finding container d81bbd1e64a3f31434241fbc652f30f0303c7dcf7902c32319def4cf09cc235b: Status 404 returned error can't find the container with id d81bbd1e64a3f31434241fbc652f30f0303c7dcf7902c32319def4cf09cc235b
Jul 28 14:27:02.795644 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:02.795570    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-z2tkf"]
Jul 28 14:27:02.799758 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:02.799736    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-z2tkf"
Jul 28 14:27:02.802785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:02.802763    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-49ffk\""
Jul 28 14:27:02.808737 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:02.808702    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-z2tkf"]
Jul 28 14:27:02.823553 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:02.823520    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bmps4" event={"ID":"2be2218c-f24d-4029-8b68-20b28b22767e","Type":"ContainerStarted","Data":"d81bbd1e64a3f31434241fbc652f30f0303c7dcf7902c32319def4cf09cc235b"}
Jul 28 14:27:02.840910 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:02.840882    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtk49\" (UniqueName: \"kubernetes.io/projected/454b46b7-039b-4848-b7cd-fc404487d431-kube-api-access-qtk49\") pod \"authorino-58cd955675-z2tkf\" (UID: \"454b46b7-039b-4848-b7cd-fc404487d431\") " pod="kuadrant-system/authorino-58cd955675-z2tkf"
Jul 28 14:27:02.941808 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:02.941766    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtk49\" (UniqueName: \"kubernetes.io/projected/454b46b7-039b-4848-b7cd-fc404487d431-kube-api-access-qtk49\") pod \"authorino-58cd955675-z2tkf\" (UID: \"454b46b7-039b-4848-b7cd-fc404487d431\") " pod="kuadrant-system/authorino-58cd955675-z2tkf"
Jul 28 14:27:02.951522 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:02.951491    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtk49\" (UniqueName: \"kubernetes.io/projected/454b46b7-039b-4848-b7cd-fc404487d431-kube-api-access-qtk49\") pod \"authorino-58cd955675-z2tkf\" (UID: \"454b46b7-039b-4848-b7cd-fc404487d431\") " pod="kuadrant-system/authorino-58cd955675-z2tkf"
Jul 28 14:27:03.109257 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:03.109216    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-z2tkf"
Jul 28 14:27:03.320104 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:03.320053    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-z2tkf"]
Jul 28 14:27:03.325936 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:27:03.325897    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod454b46b7_039b_4848_b7cd_fc404487d431.slice/crio-59348eabcae15cd0cf8e611ad5a4bc54c7617996e8b8d59fe3dfe47ab765c04f WatchSource:0}: Error finding container 59348eabcae15cd0cf8e611ad5a4bc54c7617996e8b8d59fe3dfe47ab765c04f: Status 404 returned error can't find the container with id 59348eabcae15cd0cf8e611ad5a4bc54c7617996e8b8d59fe3dfe47ab765c04f
Jul 28 14:27:03.829525 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:03.829474    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-z2tkf" event={"ID":"454b46b7-039b-4848-b7cd-fc404487d431","Type":"ContainerStarted","Data":"59348eabcae15cd0cf8e611ad5a4bc54c7617996e8b8d59fe3dfe47ab765c04f"}
Jul 28 14:27:05.844127 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:05.844072    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bmps4" event={"ID":"2be2218c-f24d-4029-8b68-20b28b22767e","Type":"ContainerStarted","Data":"e4b70c6cdd5a8dc5ff8ebb0f49a6767644a281852a11ab808bc16f90b5fa6f8e"}
Jul 28 14:27:05.844591 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:05.844197    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-bmps4"
Jul 28 14:27:05.863312 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:05.863154    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-bmps4" podStartSLOduration=1.590043122 podStartE2EDuration="4.863137658s" totalImagesPullingTime="3.273094536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:27:05.861732784 +0000 UTC m=+495.663275000" watchObservedRunningTime="2026-07-28 14:27:05.863137658 +0000 UTC m=+495.664679874"
Jul 28 14:27:07.684685 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:07.684597    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-z2tkf"]
Jul 28 14:27:07.854838 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:07.854800    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-z2tkf" event={"ID":"454b46b7-039b-4848-b7cd-fc404487d431","Type":"ContainerStarted","Data":"bcbd11f39373b1e014ee3ae7effdc86edcf8409d7cac84af89977c1cfe3946df"}
Jul 28 14:27:07.890178 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:07.890116    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-z2tkf" podStartSLOduration=1.801133509 podStartE2EDuration="5.8900819s" totalImagesPullingTime="4.088948391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:27:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:27:07.888487154 +0000 UTC m=+497.690029368" watchObservedRunningTime="2026-07-28 14:27:07.8900819 +0000 UTC m=+497.691624114"
Jul 28 14:27:08.858639 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:08.858601    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-z2tkf" podUID="454b46b7-039b-4848-b7cd-fc404487d431" containerName="authorino" containerID="cri-o://bcbd11f39373b1e014ee3ae7effdc86edcf8409d7cac84af89977c1cfe3946df" gracePeriod=30
Jul 28 14:27:09.102901 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:09.102868    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-z2tkf"
Jul 28 14:27:09.204744 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:09.204653    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtk49\" (UniqueName: \"kubernetes.io/projected/454b46b7-039b-4848-b7cd-fc404487d431-kube-api-access-qtk49\") pod \"454b46b7-039b-4848-b7cd-fc404487d431\" (UID: \"454b46b7-039b-4848-b7cd-fc404487d431\") "
Jul 28 14:27:09.206792 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:09.206739    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/454b46b7-039b-4848-b7cd-fc404487d431-kube-api-access-qtk49" (OuterVolumeSpecName: "kube-api-access-qtk49") pod "454b46b7-039b-4848-b7cd-fc404487d431" (UID: "454b46b7-039b-4848-b7cd-fc404487d431"). InnerVolumeSpecName "kube-api-access-qtk49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:27:09.305389 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:09.305351    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtk49\" (UniqueName: \"kubernetes.io/projected/454b46b7-039b-4848-b7cd-fc404487d431-kube-api-access-qtk49\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:27:09.863821 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:09.863783    2584 generic.go:356] "Generic (PLEG): container finished" podID="454b46b7-039b-4848-b7cd-fc404487d431" containerID="bcbd11f39373b1e014ee3ae7effdc86edcf8409d7cac84af89977c1cfe3946df" exitCode=0
Jul 28 14:27:09.863821 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:09.863826    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-z2tkf"
Jul 28 14:27:09.864347 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:09.863858    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-z2tkf" event={"ID":"454b46b7-039b-4848-b7cd-fc404487d431","Type":"ContainerDied","Data":"bcbd11f39373b1e014ee3ae7effdc86edcf8409d7cac84af89977c1cfe3946df"}
Jul 28 14:27:09.864347 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:09.863891    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-z2tkf" event={"ID":"454b46b7-039b-4848-b7cd-fc404487d431","Type":"ContainerDied","Data":"59348eabcae15cd0cf8e611ad5a4bc54c7617996e8b8d59fe3dfe47ab765c04f"}
Jul 28 14:27:09.864347 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:09.863908    2584 scope.go:117] "RemoveContainer" containerID="bcbd11f39373b1e014ee3ae7effdc86edcf8409d7cac84af89977c1cfe3946df"
Jul 28 14:27:09.872934 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:09.872908    2584 scope.go:117] "RemoveContainer" containerID="bcbd11f39373b1e014ee3ae7effdc86edcf8409d7cac84af89977c1cfe3946df"
Jul 28 14:27:09.873298 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:27:09.873278    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcbd11f39373b1e014ee3ae7effdc86edcf8409d7cac84af89977c1cfe3946df\": container with ID starting with bcbd11f39373b1e014ee3ae7effdc86edcf8409d7cac84af89977c1cfe3946df not found: ID does not exist" containerID="bcbd11f39373b1e014ee3ae7effdc86edcf8409d7cac84af89977c1cfe3946df"
Jul 28 14:27:09.873369 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:09.873306    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcbd11f39373b1e014ee3ae7effdc86edcf8409d7cac84af89977c1cfe3946df"} err="failed to get container status \"bcbd11f39373b1e014ee3ae7effdc86edcf8409d7cac84af89977c1cfe3946df\": rpc error: code = NotFound desc = could not find container \"bcbd11f39373b1e014ee3ae7effdc86edcf8409d7cac84af89977c1cfe3946df\": container with ID starting with bcbd11f39373b1e014ee3ae7effdc86edcf8409d7cac84af89977c1cfe3946df not found: ID does not exist"
Jul 28 14:27:09.886757 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:09.886718    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-z2tkf"]
Jul 28 14:27:09.890768 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:09.890734    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-z2tkf"]
Jul 28 14:27:10.796585 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:10.796546    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="454b46b7-039b-4848-b7cd-fc404487d431" path="/var/lib/kubelet/pods/454b46b7-039b-4848-b7cd-fc404487d431/volumes"
Jul 28 14:27:16.850314 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:16.850281    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-bmps4"
Jul 28 14:27:19.402503 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:19.402466    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bmps4"]
Jul 28 14:27:19.402887 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:19.402675    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-bmps4" podUID="2be2218c-f24d-4029-8b68-20b28b22767e" containerName="limitador" containerID="cri-o://e4b70c6cdd5a8dc5ff8ebb0f49a6767644a281852a11ab808bc16f90b5fa6f8e" gracePeriod=30
Jul 28 14:27:19.909825 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:19.909789    2584 generic.go:356] "Generic (PLEG): container finished" podID="2be2218c-f24d-4029-8b68-20b28b22767e" containerID="e4b70c6cdd5a8dc5ff8ebb0f49a6767644a281852a11ab808bc16f90b5fa6f8e" exitCode=0
Jul 28 14:27:19.910006 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:19.909871    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bmps4" event={"ID":"2be2218c-f24d-4029-8b68-20b28b22767e","Type":"ContainerDied","Data":"e4b70c6cdd5a8dc5ff8ebb0f49a6767644a281852a11ab808bc16f90b5fa6f8e"}
Jul 28 14:27:19.956414 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:19.956387    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bmps4"
Jul 28 14:27:20.101050 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:20.101027    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94zpk\" (UniqueName: \"kubernetes.io/projected/2be2218c-f24d-4029-8b68-20b28b22767e-kube-api-access-94zpk\") pod \"2be2218c-f24d-4029-8b68-20b28b22767e\" (UID: \"2be2218c-f24d-4029-8b68-20b28b22767e\") "
Jul 28 14:27:20.101247 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:20.101071    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2be2218c-f24d-4029-8b68-20b28b22767e-config-file\") pod \"2be2218c-f24d-4029-8b68-20b28b22767e\" (UID: \"2be2218c-f24d-4029-8b68-20b28b22767e\") "
Jul 28 14:27:20.101507 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:20.101484    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2be2218c-f24d-4029-8b68-20b28b22767e-config-file" (OuterVolumeSpecName: "config-file") pod "2be2218c-f24d-4029-8b68-20b28b22767e" (UID: "2be2218c-f24d-4029-8b68-20b28b22767e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:27:20.103260 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:20.103227    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2be2218c-f24d-4029-8b68-20b28b22767e-kube-api-access-94zpk" (OuterVolumeSpecName: "kube-api-access-94zpk") pod "2be2218c-f24d-4029-8b68-20b28b22767e" (UID: "2be2218c-f24d-4029-8b68-20b28b22767e"). InnerVolumeSpecName "kube-api-access-94zpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:27:20.201714 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:20.201678    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94zpk\" (UniqueName: \"kubernetes.io/projected/2be2218c-f24d-4029-8b68-20b28b22767e-kube-api-access-94zpk\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:27:20.201714 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:20.201707    2584 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2be2218c-f24d-4029-8b68-20b28b22767e-config-file\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:27:20.915692 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:20.915659    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bmps4" event={"ID":"2be2218c-f24d-4029-8b68-20b28b22767e","Type":"ContainerDied","Data":"d81bbd1e64a3f31434241fbc652f30f0303c7dcf7902c32319def4cf09cc235b"}
Jul 28 14:27:20.916151 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:20.915721    2584 scope.go:117] "RemoveContainer" containerID="e4b70c6cdd5a8dc5ff8ebb0f49a6767644a281852a11ab808bc16f90b5fa6f8e"
Jul 28 14:27:20.916151 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:20.915673    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bmps4"
Jul 28 14:27:20.935904 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:20.935871    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bmps4"]
Jul 28 14:27:20.943195 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:20.943160    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bmps4"]
Jul 28 14:27:22.795440 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:22.795405    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2be2218c-f24d-4029-8b68-20b28b22767e" path="/var/lib/kubelet/pods/2be2218c-f24d-4029-8b68-20b28b22767e/volumes"
Jul 28 14:27:28.830142 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:28.830084    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-x86gq"]
Jul 28 14:27:28.830632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:28.830563    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="454b46b7-039b-4848-b7cd-fc404487d431" containerName="authorino"
Jul 28 14:27:28.830632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:28.830580    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="454b46b7-039b-4848-b7cd-fc404487d431" containerName="authorino"
Jul 28 14:27:28.830632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:28.830599    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2be2218c-f24d-4029-8b68-20b28b22767e" containerName="limitador"
Jul 28 14:27:28.830632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:28.830607    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2be2218c-f24d-4029-8b68-20b28b22767e" containerName="limitador"
Jul 28 14:27:28.830831 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:28.830700    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="454b46b7-039b-4848-b7cd-fc404487d431" containerName="authorino"
Jul 28 14:27:28.830831 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:28.830713    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2be2218c-f24d-4029-8b68-20b28b22767e" containerName="limitador"
Jul 28 14:27:28.833923 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:28.833900    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x86gq"
Jul 28 14:27:28.836888 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:28.836865    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 14:27:28.836888 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:28.836877    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 28 14:27:28.837895 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:28.837874    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 14:27:28.837895 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:28.837893    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-pg8qf\""
Jul 28 14:27:28.846537 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:28.846515    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-x86gq"]
Jul 28 14:27:28.977781 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:28.977737    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ea3e46f5-4237-4fa5-9d99-cc58d394a04b-data\") pod \"postgres-69fb4594cf-x86gq\" (UID: \"ea3e46f5-4237-4fa5-9d99-cc58d394a04b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x86gq"
Jul 28 14:27:28.977967 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:28.977806    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v6df\" (UniqueName: \"kubernetes.io/projected/ea3e46f5-4237-4fa5-9d99-cc58d394a04b-kube-api-access-4v6df\") pod \"postgres-69fb4594cf-x86gq\" (UID: \"ea3e46f5-4237-4fa5-9d99-cc58d394a04b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x86gq"
Jul 28 14:27:29.078586 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:29.078550    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v6df\" (UniqueName: \"kubernetes.io/projected/ea3e46f5-4237-4fa5-9d99-cc58d394a04b-kube-api-access-4v6df\") pod \"postgres-69fb4594cf-x86gq\" (UID: \"ea3e46f5-4237-4fa5-9d99-cc58d394a04b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x86gq"
Jul 28 14:27:29.078758 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:29.078634    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ea3e46f5-4237-4fa5-9d99-cc58d394a04b-data\") pod \"postgres-69fb4594cf-x86gq\" (UID: \"ea3e46f5-4237-4fa5-9d99-cc58d394a04b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x86gq"
Jul 28 14:27:29.079038 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:29.079019    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ea3e46f5-4237-4fa5-9d99-cc58d394a04b-data\") pod \"postgres-69fb4594cf-x86gq\" (UID: \"ea3e46f5-4237-4fa5-9d99-cc58d394a04b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x86gq"
Jul 28 14:27:29.091283 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:29.091205    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v6df\" (UniqueName: \"kubernetes.io/projected/ea3e46f5-4237-4fa5-9d99-cc58d394a04b-kube-api-access-4v6df\") pod \"postgres-69fb4594cf-x86gq\" (UID: \"ea3e46f5-4237-4fa5-9d99-cc58d394a04b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x86gq"
Jul 28 14:27:29.147372 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:29.147315    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x86gq"
Jul 28 14:27:29.300878 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:29.300848    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-x86gq"]
Jul 28 14:27:29.303768 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:27:29.303739    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea3e46f5_4237_4fa5_9d99_cc58d394a04b.slice/crio-cfebd477a07cff23aa1a03ae27f15f6605a2446b65a0037275347ade7b0e7dea WatchSource:0}: Error finding container cfebd477a07cff23aa1a03ae27f15f6605a2446b65a0037275347ade7b0e7dea: Status 404 returned error can't find the container with id cfebd477a07cff23aa1a03ae27f15f6605a2446b65a0037275347ade7b0e7dea
Jul 28 14:27:29.954708 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:29.954668    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x86gq" event={"ID":"ea3e46f5-4237-4fa5-9d99-cc58d394a04b","Type":"ContainerStarted","Data":"cfebd477a07cff23aa1a03ae27f15f6605a2446b65a0037275347ade7b0e7dea"}
Jul 28 14:27:34.983462 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:34.983416    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x86gq" event={"ID":"ea3e46f5-4237-4fa5-9d99-cc58d394a04b","Type":"ContainerStarted","Data":"1ab770195b1ec625235c7b4ad1e4d055af2ee31880ccb3b4a383d0e95d0afff3"}
Jul 28 14:27:34.983965 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:34.983489    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x86gq"
Jul 28 14:27:35.001630 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:35.001572    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x86gq" podStartSLOduration=1.926995757 podStartE2EDuration="7.001557416s" totalImagesPullingTime="5.074561659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:27:35.000497517 +0000 UTC m=+524.802039735" watchObservedRunningTime="2026-07-28 14:27:35.001557416 +0000 UTC m=+524.803099630"
Jul 28 14:27:41.016783 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.016751    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x86gq"
Jul 28 14:27:41.620849 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.620813    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gwtj4"]
Jul 28 14:27:41.627300 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.627273    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gwtj4"
Jul 28 14:27:41.630064 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.630042    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-49ffk\""
Jul 28 14:27:41.630233 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.630195    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gwtj4"]
Jul 28 14:27:41.682955 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.682912    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zj56\" (UniqueName: \"kubernetes.io/projected/16964e6b-d3b7-49bd-94f3-b936c07f7e54-kube-api-access-4zj56\") pod \"authorino-5d4b67c446-gwtj4\" (UID: \"16964e6b-d3b7-49bd-94f3-b936c07f7e54\") " pod="kuadrant-system/authorino-5d4b67c446-gwtj4"
Jul 28 14:27:41.784327 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.784294    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zj56\" (UniqueName: \"kubernetes.io/projected/16964e6b-d3b7-49bd-94f3-b936c07f7e54-kube-api-access-4zj56\") pod \"authorino-5d4b67c446-gwtj4\" (UID: \"16964e6b-d3b7-49bd-94f3-b936c07f7e54\") " pod="kuadrant-system/authorino-5d4b67c446-gwtj4"
Jul 28 14:27:41.793394 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.793368    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zj56\" (UniqueName: \"kubernetes.io/projected/16964e6b-d3b7-49bd-94f3-b936c07f7e54-kube-api-access-4zj56\") pod \"authorino-5d4b67c446-gwtj4\" (UID: \"16964e6b-d3b7-49bd-94f3-b936c07f7e54\") " pod="kuadrant-system/authorino-5d4b67c446-gwtj4"
Jul 28 14:27:41.825532 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.825496    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gwtj4"]
Jul 28 14:27:41.825755 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.825742    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gwtj4"
Jul 28 14:27:41.858994 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.858961    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-k28vf"]
Jul 28 14:27:41.861952 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.861932    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-k28vf"
Jul 28 14:27:41.869788 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.869753    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 28 14:27:41.873837 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.873776    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-k28vf"]
Jul 28 14:27:41.885970 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.885933    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nfxb\" (UniqueName: \"kubernetes.io/projected/18c75bd2-5352-46b5-bd39-c9ae654190cf-kube-api-access-9nfxb\") pod \"authorino-76877994-k28vf\" (UID: \"18c75bd2-5352-46b5-bd39-c9ae654190cf\") " pod="kuadrant-system/authorino-76877994-k28vf"
Jul 28 14:27:41.886980 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.886919    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/18c75bd2-5352-46b5-bd39-c9ae654190cf-tls-cert\") pod \"authorino-76877994-k28vf\" (UID: \"18c75bd2-5352-46b5-bd39-c9ae654190cf\") " pod="kuadrant-system/authorino-76877994-k28vf"
Jul 28 14:27:41.902647 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.902615    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-k28vf"]
Jul 28 14:27:41.902904 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:27:41.902868    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-9nfxb tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-k28vf" podUID="18c75bd2-5352-46b5-bd39-c9ae654190cf"
Jul 28 14:27:41.944690 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.944648    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d85746f45-72bc6"]
Jul 28 14:27:41.947061 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.947038    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d85746f45-72bc6"
Jul 28 14:27:41.956956 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.956932    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d85746f45-72bc6"]
Jul 28 14:27:41.968963 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.968897    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gwtj4"]
Jul 28 14:27:41.971185 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:27:41.971146    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16964e6b_d3b7_49bd_94f3_b936c07f7e54.slice/crio-fbd5b70af74861bc056b545bc4a41288d43b9d4b58f1b066a012e4d5d7677378 WatchSource:0}: Error finding container fbd5b70af74861bc056b545bc4a41288d43b9d4b58f1b066a012e4d5d7677378: Status 404 returned error can't find the container with id fbd5b70af74861bc056b545bc4a41288d43b9d4b58f1b066a012e4d5d7677378
Jul 28 14:27:41.988366 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.988336    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nfxb\" (UniqueName: \"kubernetes.io/projected/18c75bd2-5352-46b5-bd39-c9ae654190cf-kube-api-access-9nfxb\") pod \"authorino-76877994-k28vf\" (UID: \"18c75bd2-5352-46b5-bd39-c9ae654190cf\") " pod="kuadrant-system/authorino-76877994-k28vf"
Jul 28 14:27:41.988514 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.988381    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/18c75bd2-5352-46b5-bd39-c9ae654190cf-tls-cert\") pod \"authorino-76877994-k28vf\" (UID: \"18c75bd2-5352-46b5-bd39-c9ae654190cf\") " pod="kuadrant-system/authorino-76877994-k28vf"
Jul 28 14:27:41.988514 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.988446    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/57496ac1-9947-43c8-baec-16e43902d54d-tls-cert\") pod \"authorino-5d85746f45-72bc6\" (UID: \"57496ac1-9947-43c8-baec-16e43902d54d\") " pod="kuadrant-system/authorino-5d85746f45-72bc6"
Jul 28 14:27:41.988514 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.988472    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkkn7\" (UniqueName: \"kubernetes.io/projected/57496ac1-9947-43c8-baec-16e43902d54d-kube-api-access-jkkn7\") pod \"authorino-5d85746f45-72bc6\" (UID: \"57496ac1-9947-43c8-baec-16e43902d54d\") " pod="kuadrant-system/authorino-5d85746f45-72bc6"
Jul 28 14:27:41.990819 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.990797    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/18c75bd2-5352-46b5-bd39-c9ae654190cf-tls-cert\") pod \"authorino-76877994-k28vf\" (UID: \"18c75bd2-5352-46b5-bd39-c9ae654190cf\") " pod="kuadrant-system/authorino-76877994-k28vf"
Jul 28 14:27:41.998765 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:41.998733    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nfxb\" (UniqueName: \"kubernetes.io/projected/18c75bd2-5352-46b5-bd39-c9ae654190cf-kube-api-access-9nfxb\") pod \"authorino-76877994-k28vf\" (UID: \"18c75bd2-5352-46b5-bd39-c9ae654190cf\") " pod="kuadrant-system/authorino-76877994-k28vf"
Jul 28 14:27:42.015142 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:42.015052    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gwtj4" event={"ID":"16964e6b-d3b7-49bd-94f3-b936c07f7e54","Type":"ContainerStarted","Data":"fbd5b70af74861bc056b545bc4a41288d43b9d4b58f1b066a012e4d5d7677378"}
Jul 28 14:27:42.015142 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:42.015075    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-k28vf"
Jul 28 14:27:42.022067 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:42.022042    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-k28vf"
Jul 28 14:27:42.089962 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:42.089876    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/18c75bd2-5352-46b5-bd39-c9ae654190cf-tls-cert\") pod \"18c75bd2-5352-46b5-bd39-c9ae654190cf\" (UID: \"18c75bd2-5352-46b5-bd39-c9ae654190cf\") "
Jul 28 14:27:42.089962 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:42.089940    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nfxb\" (UniqueName: \"kubernetes.io/projected/18c75bd2-5352-46b5-bd39-c9ae654190cf-kube-api-access-9nfxb\") pod \"18c75bd2-5352-46b5-bd39-c9ae654190cf\" (UID: \"18c75bd2-5352-46b5-bd39-c9ae654190cf\") "
Jul 28 14:27:42.090253 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:42.090196    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/57496ac1-9947-43c8-baec-16e43902d54d-tls-cert\") pod \"authorino-5d85746f45-72bc6\" (UID: \"57496ac1-9947-43c8-baec-16e43902d54d\") " pod="kuadrant-system/authorino-5d85746f45-72bc6"
Jul 28 14:27:42.090253 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:42.090238    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkkn7\" (UniqueName: \"kubernetes.io/projected/57496ac1-9947-43c8-baec-16e43902d54d-kube-api-access-jkkn7\") pod \"authorino-5d85746f45-72bc6\" (UID: \"57496ac1-9947-43c8-baec-16e43902d54d\") " pod="kuadrant-system/authorino-5d85746f45-72bc6"
Jul 28 14:27:42.092037 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:42.092008    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18c75bd2-5352-46b5-bd39-c9ae654190cf-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "18c75bd2-5352-46b5-bd39-c9ae654190cf" (UID: "18c75bd2-5352-46b5-bd39-c9ae654190cf"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:27:42.092037 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:42.092009    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18c75bd2-5352-46b5-bd39-c9ae654190cf-kube-api-access-9nfxb" (OuterVolumeSpecName: "kube-api-access-9nfxb") pod "18c75bd2-5352-46b5-bd39-c9ae654190cf" (UID: "18c75bd2-5352-46b5-bd39-c9ae654190cf"). InnerVolumeSpecName "kube-api-access-9nfxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:27:42.092411 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:42.092392    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/57496ac1-9947-43c8-baec-16e43902d54d-tls-cert\") pod \"authorino-5d85746f45-72bc6\" (UID: \"57496ac1-9947-43c8-baec-16e43902d54d\") " pod="kuadrant-system/authorino-5d85746f45-72bc6"
Jul 28 14:27:42.099291 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:42.099270    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkkn7\" (UniqueName: \"kubernetes.io/projected/57496ac1-9947-43c8-baec-16e43902d54d-kube-api-access-jkkn7\") pod \"authorino-5d85746f45-72bc6\" (UID: \"57496ac1-9947-43c8-baec-16e43902d54d\") " pod="kuadrant-system/authorino-5d85746f45-72bc6"
Jul 28 14:27:42.190836 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:42.190755    2584 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/18c75bd2-5352-46b5-bd39-c9ae654190cf-tls-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:27:42.190836 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:42.190778    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nfxb\" (UniqueName: \"kubernetes.io/projected/18c75bd2-5352-46b5-bd39-c9ae654190cf-kube-api-access-9nfxb\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:27:42.257554 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:42.257499    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d85746f45-72bc6"
Jul 28 14:27:42.396310 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:42.396141    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d85746f45-72bc6"]
Jul 28 14:27:42.398810 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:27:42.398779    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57496ac1_9947_43c8_baec_16e43902d54d.slice/crio-2624b71ffa8859ed235fba54b02f779265417359ab7503364c194405eb2e5d65 WatchSource:0}: Error finding container 2624b71ffa8859ed235fba54b02f779265417359ab7503364c194405eb2e5d65: Status 404 returned error can't find the container with id 2624b71ffa8859ed235fba54b02f779265417359ab7503364c194405eb2e5d65
Jul 28 14:27:43.020578 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:43.020541    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d85746f45-72bc6" event={"ID":"57496ac1-9947-43c8-baec-16e43902d54d","Type":"ContainerStarted","Data":"2624b71ffa8859ed235fba54b02f779265417359ab7503364c194405eb2e5d65"}
Jul 28 14:27:43.021903 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:43.021872    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gwtj4" event={"ID":"16964e6b-d3b7-49bd-94f3-b936c07f7e54","Type":"ContainerStarted","Data":"64d92dff4322158e3999454b6cae36cff11ba7a23146f3bb6a3a3bca8f5d1ed4"}
Jul 28 14:27:43.021903 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:43.021879    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-k28vf"
Jul 28 14:27:43.022185 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:43.021888    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-gwtj4" podUID="16964e6b-d3b7-49bd-94f3-b936c07f7e54" containerName="authorino" containerID="cri-o://64d92dff4322158e3999454b6cae36cff11ba7a23146f3bb6a3a3bca8f5d1ed4" gracePeriod=30
Jul 28 14:27:43.042636 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:43.042579    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-gwtj4" podStartSLOduration=1.727266634 podStartE2EDuration="2.042560831s" totalImagesPullingTime="315.294197ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:27:43.041030817 +0000 UTC m=+532.842573031" watchObservedRunningTime="2026-07-28 14:27:43.042560831 +0000 UTC m=+532.844103045"
Jul 28 14:27:43.069970 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:43.069880    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-k28vf"]
Jul 28 14:27:43.072850 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:43.072819    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-k28vf"]
Jul 28 14:27:43.274720 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:43.274694    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gwtj4"
Jul 28 14:27:43.301592 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:43.301561    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zj56\" (UniqueName: \"kubernetes.io/projected/16964e6b-d3b7-49bd-94f3-b936c07f7e54-kube-api-access-4zj56\") pod \"16964e6b-d3b7-49bd-94f3-b936c07f7e54\" (UID: \"16964e6b-d3b7-49bd-94f3-b936c07f7e54\") "
Jul 28 14:27:43.303610 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:43.303580    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16964e6b-d3b7-49bd-94f3-b936c07f7e54-kube-api-access-4zj56" (OuterVolumeSpecName: "kube-api-access-4zj56") pod "16964e6b-d3b7-49bd-94f3-b936c07f7e54" (UID: "16964e6b-d3b7-49bd-94f3-b936c07f7e54"). InnerVolumeSpecName "kube-api-access-4zj56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:27:43.402944 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:43.402909    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zj56\" (UniqueName: \"kubernetes.io/projected/16964e6b-d3b7-49bd-94f3-b936c07f7e54-kube-api-access-4zj56\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:27:44.027953 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:44.027917    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d85746f45-72bc6" event={"ID":"57496ac1-9947-43c8-baec-16e43902d54d","Type":"ContainerStarted","Data":"3254aebfc1b92d3052b188008304ac54dbfd54c23efe3985ce460560323bb668"}
Jul 28 14:27:44.029064 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:44.029039    2584 generic.go:356] "Generic (PLEG): container finished" podID="16964e6b-d3b7-49bd-94f3-b936c07f7e54" containerID="64d92dff4322158e3999454b6cae36cff11ba7a23146f3bb6a3a3bca8f5d1ed4" exitCode=0
Jul 28 14:27:44.029156 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:44.029072    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gwtj4"
Jul 28 14:27:44.029156 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:44.029117    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gwtj4" event={"ID":"16964e6b-d3b7-49bd-94f3-b936c07f7e54","Type":"ContainerDied","Data":"64d92dff4322158e3999454b6cae36cff11ba7a23146f3bb6a3a3bca8f5d1ed4"}
Jul 28 14:27:44.029227 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:44.029157    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gwtj4" event={"ID":"16964e6b-d3b7-49bd-94f3-b936c07f7e54","Type":"ContainerDied","Data":"fbd5b70af74861bc056b545bc4a41288d43b9d4b58f1b066a012e4d5d7677378"}
Jul 28 14:27:44.029227 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:44.029176    2584 scope.go:117] "RemoveContainer" containerID="64d92dff4322158e3999454b6cae36cff11ba7a23146f3bb6a3a3bca8f5d1ed4"
Jul 28 14:27:44.037980 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:44.037962    2584 scope.go:117] "RemoveContainer" containerID="64d92dff4322158e3999454b6cae36cff11ba7a23146f3bb6a3a3bca8f5d1ed4"
Jul 28 14:27:44.038298 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:27:44.038278    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64d92dff4322158e3999454b6cae36cff11ba7a23146f3bb6a3a3bca8f5d1ed4\": container with ID starting with 64d92dff4322158e3999454b6cae36cff11ba7a23146f3bb6a3a3bca8f5d1ed4 not found: ID does not exist" containerID="64d92dff4322158e3999454b6cae36cff11ba7a23146f3bb6a3a3bca8f5d1ed4"
Jul 28 14:27:44.038369 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:44.038312    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64d92dff4322158e3999454b6cae36cff11ba7a23146f3bb6a3a3bca8f5d1ed4"} err="failed to get container status \"64d92dff4322158e3999454b6cae36cff11ba7a23146f3bb6a3a3bca8f5d1ed4\": rpc error: code = NotFound desc = could not find container \"64d92dff4322158e3999454b6cae36cff11ba7a23146f3bb6a3a3bca8f5d1ed4\": container with ID starting with 64d92dff4322158e3999454b6cae36cff11ba7a23146f3bb6a3a3bca8f5d1ed4 not found: ID does not exist"
Jul 28 14:27:44.045276 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:44.045239    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d85746f45-72bc6" podStartSLOduration=2.35306566 podStartE2EDuration="3.045227192s" totalImagesPullingTime="692.161532ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:27:44.044909839 +0000 UTC m=+533.846452054" watchObservedRunningTime="2026-07-28 14:27:44.045227192 +0000 UTC m=+533.846769406"
Jul 28 14:27:44.061632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:44.061553    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gwtj4"]
Jul 28 14:27:44.070000 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:44.069967    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gwtj4"]
Jul 28 14:27:44.795762 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:44.795731    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16964e6b-d3b7-49bd-94f3-b936c07f7e54" path="/var/lib/kubelet/pods/16964e6b-d3b7-49bd-94f3-b936c07f7e54/volumes"
Jul 28 14:27:44.796078 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:44.796065    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18c75bd2-5352-46b5-bd39-c9ae654190cf" path="/var/lib/kubelet/pods/18c75bd2-5352-46b5-bd39-c9ae654190cf/volumes"
Jul 28 14:27:49.266808 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.266776    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f5465768b-k9gtd"]
Jul 28 14:27:49.267201 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.267086    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16964e6b-d3b7-49bd-94f3-b936c07f7e54" containerName="authorino"
Jul 28 14:27:49.267201 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.267112    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="16964e6b-d3b7-49bd-94f3-b936c07f7e54" containerName="authorino"
Jul 28 14:27:49.267201 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.267169    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="16964e6b-d3b7-49bd-94f3-b936c07f7e54" containerName="authorino"
Jul 28 14:27:49.269935 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.269918    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f5465768b-k9gtd"
Jul 28 14:27:49.272735 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.272712    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hvd5m\""
Jul 28 14:27:49.272974 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.272959    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 14:27:49.273633 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.273612    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 14:27:49.283224 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.283200    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f5465768b-k9gtd"]
Jul 28 14:27:49.354589 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.354559    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtwd6\" (UniqueName: \"kubernetes.io/projected/dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0-kube-api-access-mtwd6\") pod \"maas-controller-7f5465768b-k9gtd\" (UID: \"dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0\") " pod="opendatahub/maas-controller-7f5465768b-k9gtd"
Jul 28 14:27:49.354589 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.354594    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0-webhook-cert\") pod \"maas-controller-7f5465768b-k9gtd\" (UID: \"dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0\") " pod="opendatahub/maas-controller-7f5465768b-k9gtd"
Jul 28 14:27:49.455953 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.455922    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtwd6\" (UniqueName: \"kubernetes.io/projected/dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0-kube-api-access-mtwd6\") pod \"maas-controller-7f5465768b-k9gtd\" (UID: \"dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0\") " pod="opendatahub/maas-controller-7f5465768b-k9gtd"
Jul 28 14:27:49.455953 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.455960    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0-webhook-cert\") pod \"maas-controller-7f5465768b-k9gtd\" (UID: \"dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0\") " pod="opendatahub/maas-controller-7f5465768b-k9gtd"
Jul 28 14:27:49.458531 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.458503    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0-webhook-cert\") pod \"maas-controller-7f5465768b-k9gtd\" (UID: \"dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0\") " pod="opendatahub/maas-controller-7f5465768b-k9gtd"
Jul 28 14:27:49.464675 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.464647    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtwd6\" (UniqueName: \"kubernetes.io/projected/dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0-kube-api-access-mtwd6\") pod \"maas-controller-7f5465768b-k9gtd\" (UID: \"dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0\") " pod="opendatahub/maas-controller-7f5465768b-k9gtd"
Jul 28 14:27:49.468630 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.468605    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65db4695c7-mwhpx"]
Jul 28 14:27:49.470995 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.470964    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65db4695c7-mwhpx"
Jul 28 14:27:49.483823 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.483798    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65db4695c7-mwhpx"]
Jul 28 14:27:49.557225 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.557127    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ad89b0f-488a-4876-bbe3-295b51d5dff9-webhook-cert\") pod \"maas-controller-65db4695c7-mwhpx\" (UID: \"0ad89b0f-488a-4876-bbe3-295b51d5dff9\") " pod="opendatahub/maas-controller-65db4695c7-mwhpx"
Jul 28 14:27:49.557225 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.557170    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8w8s\" (UniqueName: \"kubernetes.io/projected/0ad89b0f-488a-4876-bbe3-295b51d5dff9-kube-api-access-n8w8s\") pod \"maas-controller-65db4695c7-mwhpx\" (UID: \"0ad89b0f-488a-4876-bbe3-295b51d5dff9\") " pod="opendatahub/maas-controller-65db4695c7-mwhpx"
Jul 28 14:27:49.581198 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.581161    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f5465768b-k9gtd"
Jul 28 14:27:49.657948 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.657916    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ad89b0f-488a-4876-bbe3-295b51d5dff9-webhook-cert\") pod \"maas-controller-65db4695c7-mwhpx\" (UID: \"0ad89b0f-488a-4876-bbe3-295b51d5dff9\") " pod="opendatahub/maas-controller-65db4695c7-mwhpx"
Jul 28 14:27:49.658133 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.657952    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8w8s\" (UniqueName: \"kubernetes.io/projected/0ad89b0f-488a-4876-bbe3-295b51d5dff9-kube-api-access-n8w8s\") pod \"maas-controller-65db4695c7-mwhpx\" (UID: \"0ad89b0f-488a-4876-bbe3-295b51d5dff9\") " pod="opendatahub/maas-controller-65db4695c7-mwhpx"
Jul 28 14:27:49.660524 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.660494    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ad89b0f-488a-4876-bbe3-295b51d5dff9-webhook-cert\") pod \"maas-controller-65db4695c7-mwhpx\" (UID: \"0ad89b0f-488a-4876-bbe3-295b51d5dff9\") " pod="opendatahub/maas-controller-65db4695c7-mwhpx"
Jul 28 14:27:49.668163 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.668056    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8w8s\" (UniqueName: \"kubernetes.io/projected/0ad89b0f-488a-4876-bbe3-295b51d5dff9-kube-api-access-n8w8s\") pod \"maas-controller-65db4695c7-mwhpx\" (UID: \"0ad89b0f-488a-4876-bbe3-295b51d5dff9\") " pod="opendatahub/maas-controller-65db4695c7-mwhpx"
Jul 28 14:27:49.716327 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.716248    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f5465768b-k9gtd"]
Jul 28 14:27:49.719065 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:27:49.719039    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd78c0fb_4c74_4a04_9e9c_d7dfa5c1dee0.slice/crio-5b41857d26b060c841ba78de29d4852d05573fa7c32bdbac9429d57cb68fc704 WatchSource:0}: Error finding container 5b41857d26b060c841ba78de29d4852d05573fa7c32bdbac9429d57cb68fc704: Status 404 returned error can't find the container with id 5b41857d26b060c841ba78de29d4852d05573fa7c32bdbac9429d57cb68fc704
Jul 28 14:27:49.781206 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.781168    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65db4695c7-mwhpx"
Jul 28 14:27:49.919207 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:49.919182    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65db4695c7-mwhpx"]
Jul 28 14:27:49.921871 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:27:49.921839    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ad89b0f_488a_4876_bbe3_295b51d5dff9.slice/crio-b63088268a5f80082dc0b667c199c916c95a8e3bcc190c4568f4b18b4825e314 WatchSource:0}: Error finding container b63088268a5f80082dc0b667c199c916c95a8e3bcc190c4568f4b18b4825e314: Status 404 returned error can't find the container with id b63088268a5f80082dc0b667c199c916c95a8e3bcc190c4568f4b18b4825e314
Jul 28 14:27:50.063364 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:50.063323    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f5465768b-k9gtd" event={"ID":"dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0","Type":"ContainerStarted","Data":"5b41857d26b060c841ba78de29d4852d05573fa7c32bdbac9429d57cb68fc704"}
Jul 28 14:27:50.064354 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:50.064330    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65db4695c7-mwhpx" event={"ID":"0ad89b0f-488a-4876-bbe3-295b51d5dff9","Type":"ContainerStarted","Data":"b63088268a5f80082dc0b667c199c916c95a8e3bcc190c4568f4b18b4825e314"}
Jul 28 14:27:52.655266 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:52.655241    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 14:27:53.081340 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:53.081240    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65db4695c7-mwhpx" event={"ID":"0ad89b0f-488a-4876-bbe3-295b51d5dff9","Type":"ContainerStarted","Data":"d6b2cdeef65508d8669cac9f6968fb03ddf49a4a4ae455b5b5117dbf8eb15086"}
Jul 28 14:27:53.081523 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:53.081453    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65db4695c7-mwhpx"
Jul 28 14:27:53.082976 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:53.082949    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f5465768b-k9gtd" event={"ID":"dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0","Type":"ContainerStarted","Data":"fe969ebd6c66dcfb9839b35b350331105d87dbfed92e427908fff7a1956b8121"}
Jul 28 14:27:53.083153 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:53.083056    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f5465768b-k9gtd"
Jul 28 14:27:53.117443 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:53.117388    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65db4695c7-mwhpx" podStartSLOduration=1.38144932 podStartE2EDuration="4.11737093s" totalImagesPullingTime="2.73592161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:27:53.115807631 +0000 UTC m=+542.917349848" watchObservedRunningTime="2026-07-28 14:27:53.11737093 +0000 UTC m=+542.918913144"
Jul 28 14:27:53.143444 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:27:53.143386    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f5465768b-k9gtd" podStartSLOduration=1.211173823 podStartE2EDuration="4.143368589s" totalImagesPullingTime="2.932194766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:27:53.142199869 +0000 UTC m=+542.943742085" watchObservedRunningTime="2026-07-28 14:27:53.143368589 +0000 UTC m=+542.944910846"
Jul 28 14:28:04.089086 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:04.089057    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65db4695c7-mwhpx"
Jul 28 14:28:04.089501 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:04.089237    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f5465768b-k9gtd"
Jul 28 14:28:04.162452 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:04.162413    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f5465768b-k9gtd"]
Jul 28 14:28:04.162687 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:04.162632    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7f5465768b-k9gtd" podUID="dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0" containerName="manager" containerID="cri-o://fe969ebd6c66dcfb9839b35b350331105d87dbfed92e427908fff7a1956b8121" gracePeriod=30
Jul 28 14:28:04.424607 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:04.424581    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f5465768b-k9gtd"
Jul 28 14:28:04.598529 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:04.598491    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0-webhook-cert\") pod \"dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0\" (UID: \"dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0\") "
Jul 28 14:28:04.598529 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:04.598542    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtwd6\" (UniqueName: \"kubernetes.io/projected/dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0-kube-api-access-mtwd6\") pod \"dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0\" (UID: \"dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0\") "
Jul 28 14:28:04.600765 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:04.600730    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0" (UID: "dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:28:04.600880 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:04.600789    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0-kube-api-access-mtwd6" (OuterVolumeSpecName: "kube-api-access-mtwd6") pod "dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0" (UID: "dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0"). InnerVolumeSpecName "kube-api-access-mtwd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:28:04.700151 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:04.700040    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0-webhook-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:28:04.700151 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:04.700072    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtwd6\" (UniqueName: \"kubernetes.io/projected/dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0-kube-api-access-mtwd6\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:28:05.137776 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:05.137740    2584 generic.go:356] "Generic (PLEG): container finished" podID="dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0" containerID="fe969ebd6c66dcfb9839b35b350331105d87dbfed92e427908fff7a1956b8121" exitCode=0
Jul 28 14:28:05.138237 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:05.137785    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f5465768b-k9gtd"
Jul 28 14:28:05.138237 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:05.137808    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f5465768b-k9gtd" event={"ID":"dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0","Type":"ContainerDied","Data":"fe969ebd6c66dcfb9839b35b350331105d87dbfed92e427908fff7a1956b8121"}
Jul 28 14:28:05.138237 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:05.137851    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f5465768b-k9gtd" event={"ID":"dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0","Type":"ContainerDied","Data":"5b41857d26b060c841ba78de29d4852d05573fa7c32bdbac9429d57cb68fc704"}
Jul 28 14:28:05.138237 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:05.137871    2584 scope.go:117] "RemoveContainer" containerID="fe969ebd6c66dcfb9839b35b350331105d87dbfed92e427908fff7a1956b8121"
Jul 28 14:28:05.146368 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:05.146344    2584 scope.go:117] "RemoveContainer" containerID="fe969ebd6c66dcfb9839b35b350331105d87dbfed92e427908fff7a1956b8121"
Jul 28 14:28:05.146718 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:28:05.146697    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe969ebd6c66dcfb9839b35b350331105d87dbfed92e427908fff7a1956b8121\": container with ID starting with fe969ebd6c66dcfb9839b35b350331105d87dbfed92e427908fff7a1956b8121 not found: ID does not exist" containerID="fe969ebd6c66dcfb9839b35b350331105d87dbfed92e427908fff7a1956b8121"
Jul 28 14:28:05.146784 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:05.146728    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe969ebd6c66dcfb9839b35b350331105d87dbfed92e427908fff7a1956b8121"} err="failed to get container status \"fe969ebd6c66dcfb9839b35b350331105d87dbfed92e427908fff7a1956b8121\": rpc error: code = NotFound desc = could not find container \"fe969ebd6c66dcfb9839b35b350331105d87dbfed92e427908fff7a1956b8121\": container with ID starting with fe969ebd6c66dcfb9839b35b350331105d87dbfed92e427908fff7a1956b8121 not found: ID does not exist"
Jul 28 14:28:05.155594 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:05.155568    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f5465768b-k9gtd"]
Jul 28 14:28:05.161695 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:05.161672    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7f5465768b-k9gtd"]
Jul 28 14:28:06.796158 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:06.796116    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0" path="/var/lib/kubelet/pods/dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0/volumes"
Jul 28 14:28:26.993142 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:26.993087    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5f4898b989-m22wz"]
Jul 28 14:28:26.993619 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:26.993569    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0" containerName="manager"
Jul 28 14:28:26.993619 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:26.993586    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0" containerName="manager"
Jul 28 14:28:26.993729 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:26.993668    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd78c0fb-4c74-4a04-9e9c-d7dfa5c1dee0" containerName="manager"
Jul 28 14:28:27.000888 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.000859    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f4898b989-m22wz"
Jul 28 14:28:27.003668 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.003646    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 28 14:28:27.003786 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.003691    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nnsjm\""
Jul 28 14:28:27.012323 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.012298    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f4898b989-m22wz"]
Jul 28 14:28:27.087162 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.087128    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcqnx\" (UniqueName: \"kubernetes.io/projected/c914ebc9-70b8-46c3-a048-6d4a8fdfb754-kube-api-access-xcqnx\") pod \"maas-api-5f4898b989-m22wz\" (UID: \"c914ebc9-70b8-46c3-a048-6d4a8fdfb754\") " pod="odh-ai-gateway-infra/maas-api-5f4898b989-m22wz"
Jul 28 14:28:27.087337 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.087172    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c914ebc9-70b8-46c3-a048-6d4a8fdfb754-openshift-service-ca\") pod \"maas-api-5f4898b989-m22wz\" (UID: \"c914ebc9-70b8-46c3-a048-6d4a8fdfb754\") " pod="odh-ai-gateway-infra/maas-api-5f4898b989-m22wz"
Jul 28 14:28:27.087337 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.087284    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c914ebc9-70b8-46c3-a048-6d4a8fdfb754-maas-api-tls\") pod \"maas-api-5f4898b989-m22wz\" (UID: \"c914ebc9-70b8-46c3-a048-6d4a8fdfb754\") " pod="odh-ai-gateway-infra/maas-api-5f4898b989-m22wz"
Jul 28 14:28:27.188521 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.188463    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c914ebc9-70b8-46c3-a048-6d4a8fdfb754-maas-api-tls\") pod \"maas-api-5f4898b989-m22wz\" (UID: \"c914ebc9-70b8-46c3-a048-6d4a8fdfb754\") " pod="odh-ai-gateway-infra/maas-api-5f4898b989-m22wz"
Jul 28 14:28:27.188720 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.188579    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcqnx\" (UniqueName: \"kubernetes.io/projected/c914ebc9-70b8-46c3-a048-6d4a8fdfb754-kube-api-access-xcqnx\") pod \"maas-api-5f4898b989-m22wz\" (UID: \"c914ebc9-70b8-46c3-a048-6d4a8fdfb754\") " pod="odh-ai-gateway-infra/maas-api-5f4898b989-m22wz"
Jul 28 14:28:27.188720 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.188606    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c914ebc9-70b8-46c3-a048-6d4a8fdfb754-openshift-service-ca\") pod \"maas-api-5f4898b989-m22wz\" (UID: \"c914ebc9-70b8-46c3-a048-6d4a8fdfb754\") " pod="odh-ai-gateway-infra/maas-api-5f4898b989-m22wz"
Jul 28 14:28:27.189225 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.189206    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c914ebc9-70b8-46c3-a048-6d4a8fdfb754-openshift-service-ca\") pod \"maas-api-5f4898b989-m22wz\" (UID: \"c914ebc9-70b8-46c3-a048-6d4a8fdfb754\") " pod="odh-ai-gateway-infra/maas-api-5f4898b989-m22wz"
Jul 28 14:28:27.190888 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.190868    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c914ebc9-70b8-46c3-a048-6d4a8fdfb754-maas-api-tls\") pod \"maas-api-5f4898b989-m22wz\" (UID: \"c914ebc9-70b8-46c3-a048-6d4a8fdfb754\") " pod="odh-ai-gateway-infra/maas-api-5f4898b989-m22wz"
Jul 28 14:28:27.198434 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.198408    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcqnx\" (UniqueName: \"kubernetes.io/projected/c914ebc9-70b8-46c3-a048-6d4a8fdfb754-kube-api-access-xcqnx\") pod \"maas-api-5f4898b989-m22wz\" (UID: \"c914ebc9-70b8-46c3-a048-6d4a8fdfb754\") " pod="odh-ai-gateway-infra/maas-api-5f4898b989-m22wz"
Jul 28 14:28:27.311044 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.310951    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f4898b989-m22wz"
Jul 28 14:28:27.456257 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.456223    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f4898b989-m22wz"]
Jul 28 14:28:27.459154 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:28:27.459123    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc914ebc9_70b8_46c3_a048_6d4a8fdfb754.slice/crio-ce50f5ee77dcef95114bcb14648a68f7e286f22684f3c96c160bf38f12da54df WatchSource:0}: Error finding container ce50f5ee77dcef95114bcb14648a68f7e286f22684f3c96c160bf38f12da54df: Status 404 returned error can't find the container with id ce50f5ee77dcef95114bcb14648a68f7e286f22684f3c96c160bf38f12da54df
Jul 28 14:28:27.880968 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.880935    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-82vf5"]
Jul 28 14:28:27.885852 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.885829    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82vf5"
Jul 28 14:28:27.891005 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.890984    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 28 14:28:27.891149 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.891055    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-fxbqw\""
Jul 28 14:28:27.901680 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.901656    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-82vf5"]
Jul 28 14:28:27.923612 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.923574    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-2hwhh"]
Jul 28 14:28:27.932266 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.932233    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-2hwhh"
Jul 28 14:28:27.945293 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.945261    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-2hwhh"]
Jul 28 14:28:27.995572 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.995537    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd5v9\" (UniqueName: \"kubernetes.io/projected/9566b9fe-f896-4d0a-a68a-823856c7aa06-kube-api-access-gd5v9\") pod \"payload-pre-processing-687dbb6cf6-82vf5\" (UID: \"9566b9fe-f896-4d0a-a68a-823856c7aa06\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82vf5"
Jul 28 14:28:27.995572 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.995577    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-2hwhh\" (UID: \"5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-2hwhh"
Jul 28 14:28:27.996027 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.995605    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9566b9fe-f896-4d0a-a68a-823856c7aa06-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-82vf5\" (UID: \"9566b9fe-f896-4d0a-a68a-823856c7aa06\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82vf5"
Jul 28 14:28:27.996027 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:27.995762    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff9nz\" (UniqueName: \"kubernetes.io/projected/5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2-kube-api-access-ff9nz\") pod \"payload-processing-5cc6cd444b-2hwhh\" (UID: \"5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-2hwhh"
Jul 28 14:28:28.096983 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:28.096949    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff9nz\" (UniqueName: \"kubernetes.io/projected/5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2-kube-api-access-ff9nz\") pod \"payload-processing-5cc6cd444b-2hwhh\" (UID: \"5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-2hwhh"
Jul 28 14:28:28.097223 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:28.096992    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd5v9\" (UniqueName: \"kubernetes.io/projected/9566b9fe-f896-4d0a-a68a-823856c7aa06-kube-api-access-gd5v9\") pod \"payload-pre-processing-687dbb6cf6-82vf5\" (UID: \"9566b9fe-f896-4d0a-a68a-823856c7aa06\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82vf5"
Jul 28 14:28:28.097223 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:28.097016    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-2hwhh\" (UID: \"5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-2hwhh"
Jul 28 14:28:28.097223 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:28.097203    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9566b9fe-f896-4d0a-a68a-823856c7aa06-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-82vf5\" (UID: \"9566b9fe-f896-4d0a-a68a-823856c7aa06\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82vf5"
Jul 28 14:28:28.097847 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:28.097824    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-2hwhh\" (UID: \"5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-2hwhh"
Jul 28 14:28:28.097915 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:28.097891    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9566b9fe-f896-4d0a-a68a-823856c7aa06-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-82vf5\" (UID: \"9566b9fe-f896-4d0a-a68a-823856c7aa06\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82vf5"
Jul 28 14:28:28.107963 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:28.107935    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff9nz\" (UniqueName: \"kubernetes.io/projected/5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2-kube-api-access-ff9nz\") pod \"payload-processing-5cc6cd444b-2hwhh\" (UID: \"5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-2hwhh"
Jul 28 14:28:28.111221 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:28.111202    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd5v9\" (UniqueName: \"kubernetes.io/projected/9566b9fe-f896-4d0a-a68a-823856c7aa06-kube-api-access-gd5v9\") pod \"payload-pre-processing-687dbb6cf6-82vf5\" (UID: \"9566b9fe-f896-4d0a-a68a-823856c7aa06\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82vf5"
Jul 28 14:28:28.196717 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:28.196633    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82vf5"
Jul 28 14:28:28.239116 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:28.239060    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f4898b989-m22wz" event={"ID":"c914ebc9-70b8-46c3-a048-6d4a8fdfb754","Type":"ContainerStarted","Data":"ce50f5ee77dcef95114bcb14648a68f7e286f22684f3c96c160bf38f12da54df"}
Jul 28 14:28:28.253379 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:28.253351    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-2hwhh"
Jul 28 14:28:28.355806 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:28.355384    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-82vf5"]
Jul 28 14:28:28.360746 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:28:28.360713    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9566b9fe_f896_4d0a_a68a_823856c7aa06.slice/crio-41c691766ba79b1ccbc0820451a69664b9a0ce9cf88c1e3a2e5a9f3d158d9300 WatchSource:0}: Error finding container 41c691766ba79b1ccbc0820451a69664b9a0ce9cf88c1e3a2e5a9f3d158d9300: Status 404 returned error can't find the container with id 41c691766ba79b1ccbc0820451a69664b9a0ce9cf88c1e3a2e5a9f3d158d9300
Jul 28 14:28:28.416410 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:28.416381    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-2hwhh"]
Jul 28 14:28:28.417803 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:28:28.417773    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e4aacc1_0e99_4a7e_81fb_04bc4ca5def2.slice/crio-9e7faeb2651cfd10f064a1f6fefec4d051fdf906e81e1f5a28304819d7a14a69 WatchSource:0}: Error finding container 9e7faeb2651cfd10f064a1f6fefec4d051fdf906e81e1f5a28304819d7a14a69: Status 404 returned error can't find the container with id 9e7faeb2651cfd10f064a1f6fefec4d051fdf906e81e1f5a28304819d7a14a69
Jul 28 14:28:29.253119 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:29.251029    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82vf5" event={"ID":"9566b9fe-f896-4d0a-a68a-823856c7aa06","Type":"ContainerStarted","Data":"41c691766ba79b1ccbc0820451a69664b9a0ce9cf88c1e3a2e5a9f3d158d9300"}
Jul 28 14:28:29.254276 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:29.254239    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-2hwhh" event={"ID":"5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2","Type":"ContainerStarted","Data":"9e7faeb2651cfd10f064a1f6fefec4d051fdf906e81e1f5a28304819d7a14a69"}
Jul 28 14:28:30.267508 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:30.267333    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f4898b989-m22wz" event={"ID":"c914ebc9-70b8-46c3-a048-6d4a8fdfb754","Type":"ContainerStarted","Data":"d5b316ad3b3fb9ba00ca2cb2b63da97100df4c083dd0f286bee052df893b8738"}
Jul 28 14:28:30.268000 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:30.267560    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5f4898b989-m22wz"
Jul 28 14:28:30.295124 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:30.295044    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5f4898b989-m22wz" podStartSLOduration=2.552699156 podStartE2EDuration="4.295026845s" totalImagesPullingTime="1.742327689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:28:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:28:30.293925073 +0000 UTC m=+580.095467284" watchObservedRunningTime="2026-07-28 14:28:30.295026845 +0000 UTC m=+580.096569060"
Jul 28 14:28:31.274219 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:31.274119    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-2hwhh" event={"ID":"5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2","Type":"ContainerStarted","Data":"6a23cd35226e62c1495efcb0c07885a025b5d6401c762e5840fe26f2a7773855"}
Jul 28 14:28:31.274219 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:31.274187    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-2hwhh"
Jul 28 14:28:31.275842 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:31.275819    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82vf5" event={"ID":"9566b9fe-f896-4d0a-a68a-823856c7aa06","Type":"ContainerStarted","Data":"0aa54b10ec876b8bdc823c333bb13fe6809b2dd29bdb50d011ced151a308b51a"}
Jul 28 14:28:31.275966 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:31.275874    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82vf5"
Jul 28 14:28:31.294879 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:31.294828    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-2hwhh" podStartSLOduration=1.697676988 podStartE2EDuration="4.294812854s" totalImagesPullingTime="2.597135866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:28:31.293941113 +0000 UTC m=+581.095483328" watchObservedRunningTime="2026-07-28 14:28:31.294812854 +0000 UTC m=+581.096355076"
Jul 28 14:28:31.316056 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:31.315992    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82vf5" podStartSLOduration=1.667893786 podStartE2EDuration="4.315976603s" totalImagesPullingTime="2.648082817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:28:31.313822286 +0000 UTC m=+581.115364504" watchObservedRunningTime="2026-07-28 14:28:31.315976603 +0000 UTC m=+581.117518818"
Jul 28 14:28:36.282046 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:36.282009    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5f4898b989-m22wz"
Jul 28 14:28:36.302197 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:36.302167    2584 generic.go:356] "Generic (PLEG): container finished" podID="0341cb69-6aaa-47f2-8009-28908c0830c7" containerID="2a835537a483b022daa1a51e50f6f482bde0f677ea1b51d080abe00fe0897e1d" exitCode=0
Jul 28 14:28:36.302384 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:36.302248    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp" event={"ID":"0341cb69-6aaa-47f2-8009-28908c0830c7","Type":"ContainerDied","Data":"2a835537a483b022daa1a51e50f6f482bde0f677ea1b51d080abe00fe0897e1d"}
Jul 28 14:28:36.302681 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:36.302664    2584 scope.go:117] "RemoveContainer" containerID="2a835537a483b022daa1a51e50f6f482bde0f677ea1b51d080abe00fe0897e1d"
Jul 28 14:28:36.304129 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:36.304061    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:28:36.304233 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:36.304168    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:28:36.304291 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:36.304234    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:28:37.309334 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:37.309298    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp" event={"ID":"0341cb69-6aaa-47f2-8009-28908c0830c7","Type":"ContainerStarted","Data":"c7e8bf93fa13a3524e82240d4de2926153b36314d142f5390f9afd874194be17"}
Jul 28 14:28:37.385906 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:37.385869    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:28:37.386210 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:37.386181    2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-c747fcbf8-4nzkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Jul 28 14:28:37.386269 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:37.386248    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp" podUID="0341cb69-6aaa-47f2-8009-28908c0830c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Jul 28 14:28:38.386372 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:38.386325    2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-c747fcbf8-4nzkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Jul 28 14:28:38.386860 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:38.386391    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp" podUID="0341cb69-6aaa-47f2-8009-28908c0830c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Jul 28 14:28:39.385864 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:39.385831    2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-c747fcbf8-4nzkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Jul 28 14:28:39.386039 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:39.385887    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp" podUID="0341cb69-6aaa-47f2-8009-28908c0830c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Jul 28 14:28:40.386718 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:40.386676    2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-c747fcbf8-4nzkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Jul 28 14:28:40.387158 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:40.386736    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp" podUID="0341cb69-6aaa-47f2-8009-28908c0830c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Jul 28 14:28:41.385934 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:41.385893    2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-c747fcbf8-4nzkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Jul 28 14:28:41.386151 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:41.385954    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp" podUID="0341cb69-6aaa-47f2-8009-28908c0830c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Jul 28 14:28:42.283702 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:42.283657    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82vf5" podUID="9566b9fe-f896-4d0a-a68a-823856c7aa06" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.37:9004: connect: connection refused"
Jul 28 14:28:42.284148 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:42.284128    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-2hwhh"
Jul 28 14:28:42.389709 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:42.389677    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:28:42.389971 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:42.389951    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:28:42.390631 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:42.390616    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c747fcbf8-4nzkp"
Jul 28 14:28:44.450504 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.450471    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"]
Jul 28 14:28:44.453770 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.453748    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.457673 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.457649    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 28 14:28:44.458259 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.458244    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 14:28:44.458542 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.458526    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 14:28:44.458617 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.458604    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2p4xs\""
Jul 28 14:28:44.475494 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.475457    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"]
Jul 28 14:28:44.554161 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.554125    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32e822e1-48a9-4b1e-9404-fc4f3571de8d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk\" (UID: \"32e822e1-48a9-4b1e-9404-fc4f3571de8d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.554539 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.554514    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32e822e1-48a9-4b1e-9404-fc4f3571de8d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk\" (UID: \"32e822e1-48a9-4b1e-9404-fc4f3571de8d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.554703 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.554677    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32e822e1-48a9-4b1e-9404-fc4f3571de8d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk\" (UID: \"32e822e1-48a9-4b1e-9404-fc4f3571de8d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.554838 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.554727    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32e822e1-48a9-4b1e-9404-fc4f3571de8d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk\" (UID: \"32e822e1-48a9-4b1e-9404-fc4f3571de8d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.554838 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.554759    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rmlt\" (UniqueName: \"kubernetes.io/projected/32e822e1-48a9-4b1e-9404-fc4f3571de8d-kube-api-access-6rmlt\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk\" (UID: \"32e822e1-48a9-4b1e-9404-fc4f3571de8d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.554838 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.554800    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32e822e1-48a9-4b1e-9404-fc4f3571de8d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk\" (UID: \"32e822e1-48a9-4b1e-9404-fc4f3571de8d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.656266 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.656210    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32e822e1-48a9-4b1e-9404-fc4f3571de8d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk\" (UID: \"32e822e1-48a9-4b1e-9404-fc4f3571de8d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.656470 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.656339    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rmlt\" (UniqueName: \"kubernetes.io/projected/32e822e1-48a9-4b1e-9404-fc4f3571de8d-kube-api-access-6rmlt\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk\" (UID: \"32e822e1-48a9-4b1e-9404-fc4f3571de8d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.656470 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.656369    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32e822e1-48a9-4b1e-9404-fc4f3571de8d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk\" (UID: \"32e822e1-48a9-4b1e-9404-fc4f3571de8d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.656470 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.656412    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32e822e1-48a9-4b1e-9404-fc4f3571de8d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk\" (UID: \"32e822e1-48a9-4b1e-9404-fc4f3571de8d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.656620 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.656483    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32e822e1-48a9-4b1e-9404-fc4f3571de8d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk\" (UID: \"32e822e1-48a9-4b1e-9404-fc4f3571de8d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.656620 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.656602    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32e822e1-48a9-4b1e-9404-fc4f3571de8d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk\" (UID: \"32e822e1-48a9-4b1e-9404-fc4f3571de8d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.656712 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.656619    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32e822e1-48a9-4b1e-9404-fc4f3571de8d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk\" (UID: \"32e822e1-48a9-4b1e-9404-fc4f3571de8d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.656760 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.656717    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32e822e1-48a9-4b1e-9404-fc4f3571de8d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk\" (UID: \"32e822e1-48a9-4b1e-9404-fc4f3571de8d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.656881 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.656858    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32e822e1-48a9-4b1e-9404-fc4f3571de8d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk\" (UID: \"32e822e1-48a9-4b1e-9404-fc4f3571de8d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.658679 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.658662    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32e822e1-48a9-4b1e-9404-fc4f3571de8d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk\" (UID: \"32e822e1-48a9-4b1e-9404-fc4f3571de8d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.658957 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.658938    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32e822e1-48a9-4b1e-9404-fc4f3571de8d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk\" (UID: \"32e822e1-48a9-4b1e-9404-fc4f3571de8d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.665773 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.665745    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rmlt\" (UniqueName: \"kubernetes.io/projected/32e822e1-48a9-4b1e-9404-fc4f3571de8d-kube-api-access-6rmlt\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk\" (UID: \"32e822e1-48a9-4b1e-9404-fc4f3571de8d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.765717 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.765628    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:44.910647 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:44.910615    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"]
Jul 28 14:28:44.912809 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:28:44.912778    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32e822e1_48a9_4b1e_9404_fc4f3571de8d.slice/crio-fff1da9ba7fa4f2184114d38e5f49fc036728c9df979c367c93b959b87e28075 WatchSource:0}: Error finding container fff1da9ba7fa4f2184114d38e5f49fc036728c9df979c367c93b959b87e28075: Status 404 returned error can't find the container with id fff1da9ba7fa4f2184114d38e5f49fc036728c9df979c367c93b959b87e28075
Jul 28 14:28:45.349298 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:45.349242    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk" event={"ID":"32e822e1-48a9-4b1e-9404-fc4f3571de8d","Type":"ContainerStarted","Data":"fff1da9ba7fa4f2184114d38e5f49fc036728c9df979c367c93b959b87e28075"}
Jul 28 14:28:46.355853 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:46.355813    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk" event={"ID":"32e822e1-48a9-4b1e-9404-fc4f3571de8d","Type":"ContainerStarted","Data":"f638645fcbf6a510a317c65834ba373cce28894378a4f25f47486712edb3eab1"}
Jul 28 14:28:46.378779 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:46.378722    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk" podStartSLOduration=1.083551121 podStartE2EDuration="2.378704122s" totalImagesPullingTime="1.295153001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:28:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:28:46.376924134 +0000 UTC m=+596.178466348" watchObservedRunningTime="2026-07-28 14:28:46.378704122 +0000 UTC m=+596.180246383"
Jul 28 14:28:52.282179 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:52.282151    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-82vf5"
Jul 28 14:28:54.766311 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:54.766268    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:54.766723 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:54.766348    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:54.779305 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:54.779273    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:55.408393 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.408358    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk"
Jul 28 14:28:55.462622 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.462589    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"]
Jul 28 14:28:55.467571 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.467547    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.470526 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.470501    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 28 14:28:55.478757 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.478733    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"]
Jul 28 14:28:55.560230 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.560192    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a481bb07-b3a5-4a57-ada0-788e5912e506-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-24bm6\" (UID: \"a481bb07-b3a5-4a57-ada0-788e5912e506\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.560230 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.560232    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a481bb07-b3a5-4a57-ada0-788e5912e506-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-24bm6\" (UID: \"a481bb07-b3a5-4a57-ada0-788e5912e506\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.560461 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.560333    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a481bb07-b3a5-4a57-ada0-788e5912e506-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-24bm6\" (UID: \"a481bb07-b3a5-4a57-ada0-788e5912e506\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.560461 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.560446    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a481bb07-b3a5-4a57-ada0-788e5912e506-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-24bm6\" (UID: \"a481bb07-b3a5-4a57-ada0-788e5912e506\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.560575 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.560471    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a481bb07-b3a5-4a57-ada0-788e5912e506-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-24bm6\" (UID: \"a481bb07-b3a5-4a57-ada0-788e5912e506\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.560627 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.560565    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcb48\" (UniqueName: \"kubernetes.io/projected/a481bb07-b3a5-4a57-ada0-788e5912e506-kube-api-access-tcb48\") pod \"e2e-distinct-simulated-kserve-59975d69f9-24bm6\" (UID: \"a481bb07-b3a5-4a57-ada0-788e5912e506\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.661918 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.661823    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a481bb07-b3a5-4a57-ada0-788e5912e506-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-24bm6\" (UID: \"a481bb07-b3a5-4a57-ada0-788e5912e506\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.661918 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.661879    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a481bb07-b3a5-4a57-ada0-788e5912e506-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-24bm6\" (UID: \"a481bb07-b3a5-4a57-ada0-788e5912e506\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.662150 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.661994    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a481bb07-b3a5-4a57-ada0-788e5912e506-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-24bm6\" (UID: \"a481bb07-b3a5-4a57-ada0-788e5912e506\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.662150 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.662067    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcb48\" (UniqueName: \"kubernetes.io/projected/a481bb07-b3a5-4a57-ada0-788e5912e506-kube-api-access-tcb48\") pod \"e2e-distinct-simulated-kserve-59975d69f9-24bm6\" (UID: \"a481bb07-b3a5-4a57-ada0-788e5912e506\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.662224 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.662159    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a481bb07-b3a5-4a57-ada0-788e5912e506-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-24bm6\" (UID: \"a481bb07-b3a5-4a57-ada0-788e5912e506\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.662224 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.662189    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a481bb07-b3a5-4a57-ada0-788e5912e506-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-24bm6\" (UID: \"a481bb07-b3a5-4a57-ada0-788e5912e506\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.662314 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.662281    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a481bb07-b3a5-4a57-ada0-788e5912e506-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-24bm6\" (UID: \"a481bb07-b3a5-4a57-ada0-788e5912e506\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.662377 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.662311    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a481bb07-b3a5-4a57-ada0-788e5912e506-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-24bm6\" (UID: \"a481bb07-b3a5-4a57-ada0-788e5912e506\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.662492 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.662476    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a481bb07-b3a5-4a57-ada0-788e5912e506-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-24bm6\" (UID: \"a481bb07-b3a5-4a57-ada0-788e5912e506\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.664483 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.664451    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a481bb07-b3a5-4a57-ada0-788e5912e506-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-24bm6\" (UID: \"a481bb07-b3a5-4a57-ada0-788e5912e506\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.664650 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.664631    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a481bb07-b3a5-4a57-ada0-788e5912e506-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-24bm6\" (UID: \"a481bb07-b3a5-4a57-ada0-788e5912e506\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.671141 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.671110    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcb48\" (UniqueName: \"kubernetes.io/projected/a481bb07-b3a5-4a57-ada0-788e5912e506-kube-api-access-tcb48\") pod \"e2e-distinct-simulated-kserve-59975d69f9-24bm6\" (UID: \"a481bb07-b3a5-4a57-ada0-788e5912e506\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.780254 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.780217    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:28:55.919732 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:55.919700    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"]
Jul 28 14:28:55.921084 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:28:55.921055    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda481bb07_b3a5_4a57_ada0_788e5912e506.slice/crio-55e8453f0bc43f40b65def051261c942c6fa830e8b3bc04182d97a6386373a6e WatchSource:0}: Error finding container 55e8453f0bc43f40b65def051261c942c6fa830e8b3bc04182d97a6386373a6e: Status 404 returned error can't find the container with id 55e8453f0bc43f40b65def051261c942c6fa830e8b3bc04182d97a6386373a6e
Jul 28 14:28:56.406049 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:56.406005    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6" event={"ID":"a481bb07-b3a5-4a57-ada0-788e5912e506","Type":"ContainerStarted","Data":"74526357fc83a46a1d031bafd55caaba924620a846cde36161529c7884b146a9"}
Jul 28 14:28:56.406272 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:56.406076    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6" event={"ID":"a481bb07-b3a5-4a57-ada0-788e5912e506","Type":"ContainerStarted","Data":"55e8453f0bc43f40b65def051261c942c6fa830e8b3bc04182d97a6386373a6e"}
Jul 28 14:28:56.427424 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:28:56.427292    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6" podStartSLOduration=1.211774825 podStartE2EDuration="1.427268153s" totalImagesPullingTime="215.493328ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:28:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:28:56.426356715 +0000 UTC m=+606.227898933" watchObservedRunningTime="2026-07-28 14:28:56.427268153 +0000 UTC m=+606.228810369"
Jul 28 14:29:05.781166 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:05.781126    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:29:05.781614 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:05.781186    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:29:05.793772 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:05.793747    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:29:06.462733 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:06.462702    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-24bm6"
Jul 28 14:29:43.771011 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:43.770975    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"]
Jul 28 14:29:43.775904 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:43.775882    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:43.778479 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:43.778460    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 28 14:29:43.788771 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:43.788743    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"]
Jul 28 14:29:43.901438 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:43.901396    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5ce7fab-53de-49d9-80bb-66d7fe5843ad-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p86xh\" (UID: \"a5ce7fab-53de-49d9-80bb-66d7fe5843ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:43.901438 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:43.901440    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ce7fab-53de-49d9-80bb-66d7fe5843ad-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p86xh\" (UID: \"a5ce7fab-53de-49d9-80bb-66d7fe5843ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:43.901655 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:43.901459    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75pzc\" (UniqueName: \"kubernetes.io/projected/a5ce7fab-53de-49d9-80bb-66d7fe5843ad-kube-api-access-75pzc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p86xh\" (UID: \"a5ce7fab-53de-49d9-80bb-66d7fe5843ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:43.901655 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:43.901620    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5ce7fab-53de-49d9-80bb-66d7fe5843ad-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p86xh\" (UID: \"a5ce7fab-53de-49d9-80bb-66d7fe5843ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:43.901726 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:43.901668    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5ce7fab-53de-49d9-80bb-66d7fe5843ad-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p86xh\" (UID: \"a5ce7fab-53de-49d9-80bb-66d7fe5843ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:43.901726 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:43.901685    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5ce7fab-53de-49d9-80bb-66d7fe5843ad-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p86xh\" (UID: \"a5ce7fab-53de-49d9-80bb-66d7fe5843ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:44.002436 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:44.002396    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5ce7fab-53de-49d9-80bb-66d7fe5843ad-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p86xh\" (UID: \"a5ce7fab-53de-49d9-80bb-66d7fe5843ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:44.002621 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:44.002450    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5ce7fab-53de-49d9-80bb-66d7fe5843ad-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p86xh\" (UID: \"a5ce7fab-53de-49d9-80bb-66d7fe5843ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:44.002621 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:44.002568    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5ce7fab-53de-49d9-80bb-66d7fe5843ad-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p86xh\" (UID: \"a5ce7fab-53de-49d9-80bb-66d7fe5843ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:44.002740 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:44.002627    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5ce7fab-53de-49d9-80bb-66d7fe5843ad-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p86xh\" (UID: \"a5ce7fab-53de-49d9-80bb-66d7fe5843ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:44.002740 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:44.002656    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ce7fab-53de-49d9-80bb-66d7fe5843ad-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p86xh\" (UID: \"a5ce7fab-53de-49d9-80bb-66d7fe5843ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:44.002740 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:44.002686    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75pzc\" (UniqueName: \"kubernetes.io/projected/a5ce7fab-53de-49d9-80bb-66d7fe5843ad-kube-api-access-75pzc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p86xh\" (UID: \"a5ce7fab-53de-49d9-80bb-66d7fe5843ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:44.002884 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:44.002799    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5ce7fab-53de-49d9-80bb-66d7fe5843ad-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p86xh\" (UID: \"a5ce7fab-53de-49d9-80bb-66d7fe5843ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:44.002884 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:44.002850    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5ce7fab-53de-49d9-80bb-66d7fe5843ad-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p86xh\" (UID: \"a5ce7fab-53de-49d9-80bb-66d7fe5843ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:44.002988 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:44.002933    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5ce7fab-53de-49d9-80bb-66d7fe5843ad-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p86xh\" (UID: \"a5ce7fab-53de-49d9-80bb-66d7fe5843ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:44.004797 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:44.004770    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5ce7fab-53de-49d9-80bb-66d7fe5843ad-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p86xh\" (UID: \"a5ce7fab-53de-49d9-80bb-66d7fe5843ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:44.005117 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:44.005072    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ce7fab-53de-49d9-80bb-66d7fe5843ad-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p86xh\" (UID: \"a5ce7fab-53de-49d9-80bb-66d7fe5843ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:44.011423 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:44.011399    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75pzc\" (UniqueName: \"kubernetes.io/projected/a5ce7fab-53de-49d9-80bb-66d7fe5843ad-kube-api-access-75pzc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p86xh\" (UID: \"a5ce7fab-53de-49d9-80bb-66d7fe5843ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:44.086640 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:44.086551    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:44.223797 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:44.223682    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"]
Jul 28 14:29:44.226051 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:29:44.226025    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5ce7fab_53de_49d9_80bb_66d7fe5843ad.slice/crio-e3c5cebd8c4876f933b35e730175675bbcf7f1ef6e41b2a07dd28d38981c904d WatchSource:0}: Error finding container e3c5cebd8c4876f933b35e730175675bbcf7f1ef6e41b2a07dd28d38981c904d: Status 404 returned error can't find the container with id e3c5cebd8c4876f933b35e730175675bbcf7f1ef6e41b2a07dd28d38981c904d
Jul 28 14:29:44.227346 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:44.227330    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 14:29:44.627494 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:44.627464    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh" event={"ID":"a5ce7fab-53de-49d9-80bb-66d7fe5843ad","Type":"ContainerStarted","Data":"e21425d9bfe6b7a392686a1767704b558a3a47645c32652a4bbd68d7b9a7ddcc"}
Jul 28 14:29:44.627663 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:44.627517    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh" event={"ID":"a5ce7fab-53de-49d9-80bb-66d7fe5843ad","Type":"ContainerStarted","Data":"e3c5cebd8c4876f933b35e730175675bbcf7f1ef6e41b2a07dd28d38981c904d"}
Jul 28 14:29:44.648414 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:44.648358    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh" podStartSLOduration=1.428638265 podStartE2EDuration="1.648339498s" totalImagesPullingTime="219.701233ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:29:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:29:44.646506811 +0000 UTC m=+654.448049050" watchObservedRunningTime="2026-07-28 14:29:44.648339498 +0000 UTC m=+654.449881713"
Jul 28 14:29:54.087286 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:54.087244    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:54.087756 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:54.087315    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:54.100054 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:54.100029    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:29:54.687135 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:29:54.687086    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p86xh"
Jul 28 14:31:00.833431 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:00.833334    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d85746f45-72bc6"]
Jul 28 14:31:00.834017 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:00.833562    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d85746f45-72bc6" podUID="57496ac1-9947-43c8-baec-16e43902d54d" containerName="authorino" containerID="cri-o://3254aebfc1b92d3052b188008304ac54dbfd54c23efe3985ce460560323bb668" gracePeriod=30
Jul 28 14:31:00.980494 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:00.980440    2584 generic.go:356] "Generic (PLEG): container finished" podID="57496ac1-9947-43c8-baec-16e43902d54d" containerID="3254aebfc1b92d3052b188008304ac54dbfd54c23efe3985ce460560323bb668" exitCode=0
Jul 28 14:31:00.980695 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:00.980500    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d85746f45-72bc6" event={"ID":"57496ac1-9947-43c8-baec-16e43902d54d","Type":"ContainerDied","Data":"3254aebfc1b92d3052b188008304ac54dbfd54c23efe3985ce460560323bb668"}
Jul 28 14:31:01.093051 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:01.092980    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d85746f45-72bc6"
Jul 28 14:31:01.179869 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:01.179829    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/57496ac1-9947-43c8-baec-16e43902d54d-tls-cert\") pod \"57496ac1-9947-43c8-baec-16e43902d54d\" (UID: \"57496ac1-9947-43c8-baec-16e43902d54d\") "
Jul 28 14:31:01.179869 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:01.179874    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkkn7\" (UniqueName: \"kubernetes.io/projected/57496ac1-9947-43c8-baec-16e43902d54d-kube-api-access-jkkn7\") pod \"57496ac1-9947-43c8-baec-16e43902d54d\" (UID: \"57496ac1-9947-43c8-baec-16e43902d54d\") "
Jul 28 14:31:01.181946 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:01.181913    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57496ac1-9947-43c8-baec-16e43902d54d-kube-api-access-jkkn7" (OuterVolumeSpecName: "kube-api-access-jkkn7") pod "57496ac1-9947-43c8-baec-16e43902d54d" (UID: "57496ac1-9947-43c8-baec-16e43902d54d"). InnerVolumeSpecName "kube-api-access-jkkn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:31:01.190718 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:01.190684    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57496ac1-9947-43c8-baec-16e43902d54d-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "57496ac1-9947-43c8-baec-16e43902d54d" (UID: "57496ac1-9947-43c8-baec-16e43902d54d"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:31:01.280573 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:01.280541    2584 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/57496ac1-9947-43c8-baec-16e43902d54d-tls-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:31:01.280573 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:01.280571    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkkn7\" (UniqueName: \"kubernetes.io/projected/57496ac1-9947-43c8-baec-16e43902d54d-kube-api-access-jkkn7\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:31:01.986343 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:01.986307    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d85746f45-72bc6"
Jul 28 14:31:01.986785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:01.986366    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d85746f45-72bc6" event={"ID":"57496ac1-9947-43c8-baec-16e43902d54d","Type":"ContainerDied","Data":"2624b71ffa8859ed235fba54b02f779265417359ab7503364c194405eb2e5d65"}
Jul 28 14:31:01.986785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:01.986422    2584 scope.go:117] "RemoveContainer" containerID="3254aebfc1b92d3052b188008304ac54dbfd54c23efe3985ce460560323bb668"
Jul 28 14:31:02.016458 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:02.016420    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d85746f45-72bc6"]
Jul 28 14:31:02.020787 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:02.020753    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d85746f45-72bc6"]
Jul 28 14:31:02.797588 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:02.797550    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57496ac1-9947-43c8-baec-16e43902d54d" path="/var/lib/kubelet/pods/57496ac1-9947-43c8-baec-16e43902d54d/volumes"
Jul 28 14:31:52.016248 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:52.016199    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65db4695c7-mwhpx"]
Jul 28 14:31:52.016716 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:52.016402    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-65db4695c7-mwhpx" podUID="0ad89b0f-488a-4876-bbe3-295b51d5dff9" containerName="manager" containerID="cri-o://d6b2cdeef65508d8669cac9f6968fb03ddf49a4a4ae455b5b5117dbf8eb15086" gracePeriod=30
Jul 28 14:31:52.221346 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:52.221309    2584 generic.go:356] "Generic (PLEG): container finished" podID="0ad89b0f-488a-4876-bbe3-295b51d5dff9" containerID="d6b2cdeef65508d8669cac9f6968fb03ddf49a4a4ae455b5b5117dbf8eb15086" exitCode=0
Jul 28 14:31:52.221507 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:52.221374    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65db4695c7-mwhpx" event={"ID":"0ad89b0f-488a-4876-bbe3-295b51d5dff9","Type":"ContainerDied","Data":"d6b2cdeef65508d8669cac9f6968fb03ddf49a4a4ae455b5b5117dbf8eb15086"}
Jul 28 14:31:52.272221 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:52.272157    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65db4695c7-mwhpx"
Jul 28 14:31:52.315476 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:52.315447    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ad89b0f-488a-4876-bbe3-295b51d5dff9-webhook-cert\") pod \"0ad89b0f-488a-4876-bbe3-295b51d5dff9\" (UID: \"0ad89b0f-488a-4876-bbe3-295b51d5dff9\") "
Jul 28 14:31:52.315629 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:52.315487    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8w8s\" (UniqueName: \"kubernetes.io/projected/0ad89b0f-488a-4876-bbe3-295b51d5dff9-kube-api-access-n8w8s\") pod \"0ad89b0f-488a-4876-bbe3-295b51d5dff9\" (UID: \"0ad89b0f-488a-4876-bbe3-295b51d5dff9\") "
Jul 28 14:31:52.317641 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:52.317601    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ad89b0f-488a-4876-bbe3-295b51d5dff9-kube-api-access-n8w8s" (OuterVolumeSpecName: "kube-api-access-n8w8s") pod "0ad89b0f-488a-4876-bbe3-295b51d5dff9" (UID: "0ad89b0f-488a-4876-bbe3-295b51d5dff9"). InnerVolumeSpecName "kube-api-access-n8w8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:31:52.317755 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:52.317663    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ad89b0f-488a-4876-bbe3-295b51d5dff9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0ad89b0f-488a-4876-bbe3-295b51d5dff9" (UID: "0ad89b0f-488a-4876-bbe3-295b51d5dff9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:31:52.416236 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:52.416196    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ad89b0f-488a-4876-bbe3-295b51d5dff9-webhook-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:31:52.416236 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:52.416225    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8w8s\" (UniqueName: \"kubernetes.io/projected/0ad89b0f-488a-4876-bbe3-295b51d5dff9-kube-api-access-n8w8s\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:31:53.227009 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:53.226969    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65db4695c7-mwhpx" event={"ID":"0ad89b0f-488a-4876-bbe3-295b51d5dff9","Type":"ContainerDied","Data":"b63088268a5f80082dc0b667c199c916c95a8e3bcc190c4568f4b18b4825e314"}
Jul 28 14:31:53.227455 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:53.227040    2584 scope.go:117] "RemoveContainer" containerID="d6b2cdeef65508d8669cac9f6968fb03ddf49a4a4ae455b5b5117dbf8eb15086"
Jul 28 14:31:53.227455 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:53.226985    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65db4695c7-mwhpx"
Jul 28 14:31:53.245650 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:53.245618    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65db4695c7-mwhpx"]
Jul 28 14:31:53.251968 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:53.251934    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-65db4695c7-mwhpx"]
Jul 28 14:31:54.797048 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:31:54.797014    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ad89b0f-488a-4876-bbe3-295b51d5dff9" path="/var/lib/kubelet/pods/0ad89b0f-488a-4876-bbe3-295b51d5dff9/volumes"
Jul 28 14:32:54.133114 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.133059    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65db4695c7-wvcfq"]
Jul 28 14:32:54.133539 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.133466    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57496ac1-9947-43c8-baec-16e43902d54d" containerName="authorino"
Jul 28 14:32:54.133539 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.133478    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="57496ac1-9947-43c8-baec-16e43902d54d" containerName="authorino"
Jul 28 14:32:54.133539 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.133496    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ad89b0f-488a-4876-bbe3-295b51d5dff9" containerName="manager"
Jul 28 14:32:54.133539 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.133501    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ad89b0f-488a-4876-bbe3-295b51d5dff9" containerName="manager"
Jul 28 14:32:54.133672 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.133558    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ad89b0f-488a-4876-bbe3-295b51d5dff9" containerName="manager"
Jul 28 14:32:54.133672 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.133567    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="57496ac1-9947-43c8-baec-16e43902d54d" containerName="authorino"
Jul 28 14:32:54.137786 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.137766    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65db4695c7-wvcfq"
Jul 28 14:32:54.141380 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.141356    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 14:32:54.141535 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.141356    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 14:32:54.141535 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.141356    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hvd5m\""
Jul 28 14:32:54.147254 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.147232    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65db4695c7-wvcfq"]
Jul 28 14:32:54.179793 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.179757    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d1e281da-9804-49ea-b9c0-a67fd3edb09d-webhook-cert\") pod \"maas-controller-65db4695c7-wvcfq\" (UID: \"d1e281da-9804-49ea-b9c0-a67fd3edb09d\") " pod="opendatahub/maas-controller-65db4695c7-wvcfq"
Jul 28 14:32:54.179793 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.179798    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l9pv\" (UniqueName: \"kubernetes.io/projected/d1e281da-9804-49ea-b9c0-a67fd3edb09d-kube-api-access-2l9pv\") pod \"maas-controller-65db4695c7-wvcfq\" (UID: \"d1e281da-9804-49ea-b9c0-a67fd3edb09d\") " pod="opendatahub/maas-controller-65db4695c7-wvcfq"
Jul 28 14:32:54.280897 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.280856    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d1e281da-9804-49ea-b9c0-a67fd3edb09d-webhook-cert\") pod \"maas-controller-65db4695c7-wvcfq\" (UID: \"d1e281da-9804-49ea-b9c0-a67fd3edb09d\") " pod="opendatahub/maas-controller-65db4695c7-wvcfq"
Jul 28 14:32:54.281083 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.280903    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l9pv\" (UniqueName: \"kubernetes.io/projected/d1e281da-9804-49ea-b9c0-a67fd3edb09d-kube-api-access-2l9pv\") pod \"maas-controller-65db4695c7-wvcfq\" (UID: \"d1e281da-9804-49ea-b9c0-a67fd3edb09d\") " pod="opendatahub/maas-controller-65db4695c7-wvcfq"
Jul 28 14:32:54.283398 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.283374    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d1e281da-9804-49ea-b9c0-a67fd3edb09d-webhook-cert\") pod \"maas-controller-65db4695c7-wvcfq\" (UID: \"d1e281da-9804-49ea-b9c0-a67fd3edb09d\") " pod="opendatahub/maas-controller-65db4695c7-wvcfq"
Jul 28 14:32:54.289708 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.289678    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l9pv\" (UniqueName: \"kubernetes.io/projected/d1e281da-9804-49ea-b9c0-a67fd3edb09d-kube-api-access-2l9pv\") pod \"maas-controller-65db4695c7-wvcfq\" (UID: \"d1e281da-9804-49ea-b9c0-a67fd3edb09d\") " pod="opendatahub/maas-controller-65db4695c7-wvcfq"
Jul 28 14:32:54.449173 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.449059    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65db4695c7-wvcfq"
Jul 28 14:32:54.587144 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:54.587106    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65db4695c7-wvcfq"]
Jul 28 14:32:54.590111 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:32:54.590069    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1e281da_9804_49ea_b9c0_a67fd3edb09d.slice/crio-e53b5db5556ff2f2e3ff6a2a5bbf4403421383b060f390ce19dd0e336c08a81b WatchSource:0}: Error finding container e53b5db5556ff2f2e3ff6a2a5bbf4403421383b060f390ce19dd0e336c08a81b: Status 404 returned error can't find the container with id e53b5db5556ff2f2e3ff6a2a5bbf4403421383b060f390ce19dd0e336c08a81b
Jul 28 14:32:55.513146 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:55.513080    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65db4695c7-wvcfq" event={"ID":"d1e281da-9804-49ea-b9c0-a67fd3edb09d","Type":"ContainerStarted","Data":"e77be7a97cec5b903e558cb1f405dacca65d668b1ea61c7e5a582bb2121cd9b1"}
Jul 28 14:32:55.513599 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:55.513157    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65db4695c7-wvcfq" event={"ID":"d1e281da-9804-49ea-b9c0-a67fd3edb09d","Type":"ContainerStarted","Data":"e53b5db5556ff2f2e3ff6a2a5bbf4403421383b060f390ce19dd0e336c08a81b"}
Jul 28 14:32:55.513599 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:55.513181    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65db4695c7-wvcfq"
Jul 28 14:32:55.531479 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:32:55.531428    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65db4695c7-wvcfq" podStartSLOduration=1.061558882 podStartE2EDuration="1.53141177s" totalImagesPullingTime="469.852888ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:32:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:32:55.531221822 +0000 UTC m=+845.332764037" watchObservedRunningTime="2026-07-28 14:32:55.53141177 +0000 UTC m=+845.332953984"
Jul 28 14:33:06.521046 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:06.521012    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65db4695c7-wvcfq"
Jul 28 14:33:11.169132 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:11.169083    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65db4695c7-wvcfq"]
Jul 28 14:33:11.169585 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:11.169299    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-65db4695c7-wvcfq" podUID="d1e281da-9804-49ea-b9c0-a67fd3edb09d" containerName="manager" containerID="cri-o://e77be7a97cec5b903e558cb1f405dacca65d668b1ea61c7e5a582bb2121cd9b1" gracePeriod=30
Jul 28 14:33:11.424513 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:11.424454    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65db4695c7-wvcfq"
Jul 28 14:33:11.543201 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:11.543165    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d1e281da-9804-49ea-b9c0-a67fd3edb09d-webhook-cert\") pod \"d1e281da-9804-49ea-b9c0-a67fd3edb09d\" (UID: \"d1e281da-9804-49ea-b9c0-a67fd3edb09d\") "
Jul 28 14:33:11.543390 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:11.543232    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2l9pv\" (UniqueName: \"kubernetes.io/projected/d1e281da-9804-49ea-b9c0-a67fd3edb09d-kube-api-access-2l9pv\") pod \"d1e281da-9804-49ea-b9c0-a67fd3edb09d\" (UID: \"d1e281da-9804-49ea-b9c0-a67fd3edb09d\") "
Jul 28 14:33:11.545293 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:11.545263    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1e281da-9804-49ea-b9c0-a67fd3edb09d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d1e281da-9804-49ea-b9c0-a67fd3edb09d" (UID: "d1e281da-9804-49ea-b9c0-a67fd3edb09d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:33:11.545421 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:11.545313    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1e281da-9804-49ea-b9c0-a67fd3edb09d-kube-api-access-2l9pv" (OuterVolumeSpecName: "kube-api-access-2l9pv") pod "d1e281da-9804-49ea-b9c0-a67fd3edb09d" (UID: "d1e281da-9804-49ea-b9c0-a67fd3edb09d"). InnerVolumeSpecName "kube-api-access-2l9pv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:33:11.588466 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:11.588431    2584 generic.go:356] "Generic (PLEG): container finished" podID="d1e281da-9804-49ea-b9c0-a67fd3edb09d" containerID="e77be7a97cec5b903e558cb1f405dacca65d668b1ea61c7e5a582bb2121cd9b1" exitCode=0
Jul 28 14:33:11.588765 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:11.588478    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65db4695c7-wvcfq" event={"ID":"d1e281da-9804-49ea-b9c0-a67fd3edb09d","Type":"ContainerDied","Data":"e77be7a97cec5b903e558cb1f405dacca65d668b1ea61c7e5a582bb2121cd9b1"}
Jul 28 14:33:11.588765 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:11.588524    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65db4695c7-wvcfq" event={"ID":"d1e281da-9804-49ea-b9c0-a67fd3edb09d","Type":"ContainerDied","Data":"e53b5db5556ff2f2e3ff6a2a5bbf4403421383b060f390ce19dd0e336c08a81b"}
Jul 28 14:33:11.588765 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:11.588547    2584 scope.go:117] "RemoveContainer" containerID="e77be7a97cec5b903e558cb1f405dacca65d668b1ea61c7e5a582bb2121cd9b1"
Jul 28 14:33:11.588765 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:11.588481    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65db4695c7-wvcfq"
Jul 28 14:33:11.598169 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:11.598147    2584 scope.go:117] "RemoveContainer" containerID="e77be7a97cec5b903e558cb1f405dacca65d668b1ea61c7e5a582bb2121cd9b1"
Jul 28 14:33:11.598448 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:33:11.598426    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e77be7a97cec5b903e558cb1f405dacca65d668b1ea61c7e5a582bb2121cd9b1\": container with ID starting with e77be7a97cec5b903e558cb1f405dacca65d668b1ea61c7e5a582bb2121cd9b1 not found: ID does not exist" containerID="e77be7a97cec5b903e558cb1f405dacca65d668b1ea61c7e5a582bb2121cd9b1"
Jul 28 14:33:11.598498 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:11.598459    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e77be7a97cec5b903e558cb1f405dacca65d668b1ea61c7e5a582bb2121cd9b1"} err="failed to get container status \"e77be7a97cec5b903e558cb1f405dacca65d668b1ea61c7e5a582bb2121cd9b1\": rpc error: code = NotFound desc = could not find container \"e77be7a97cec5b903e558cb1f405dacca65d668b1ea61c7e5a582bb2121cd9b1\": container with ID starting with e77be7a97cec5b903e558cb1f405dacca65d668b1ea61c7e5a582bb2121cd9b1 not found: ID does not exist"
Jul 28 14:33:11.610780 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:11.610749    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65db4695c7-wvcfq"]
Jul 28 14:33:11.614661 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:11.614635    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-65db4695c7-wvcfq"]
Jul 28 14:33:11.644225 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:11.644193    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2l9pv\" (UniqueName: \"kubernetes.io/projected/d1e281da-9804-49ea-b9c0-a67fd3edb09d-kube-api-access-2l9pv\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:33:11.644225 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:11.644223    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d1e281da-9804-49ea-b9c0-a67fd3edb09d-webhook-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:33:12.515031 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:12.514998    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65db4695c7-z9h72"]
Jul 28 14:33:12.515462 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:12.515403    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1e281da-9804-49ea-b9c0-a67fd3edb09d" containerName="manager"
Jul 28 14:33:12.515462 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:12.515416    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1e281da-9804-49ea-b9c0-a67fd3edb09d" containerName="manager"
Jul 28 14:33:12.515533 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:12.515474    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1e281da-9804-49ea-b9c0-a67fd3edb09d" containerName="manager"
Jul 28 14:33:12.519846 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:12.519829    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65db4695c7-z9h72"
Jul 28 14:33:12.524764 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:12.524727    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 14:33:12.524764 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:12.524759    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 14:33:12.535706 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:12.535679    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hvd5m\""
Jul 28 14:33:12.538122 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:12.538081    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65db4695c7-z9h72"]
Jul 28 14:33:12.655420 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:12.655379    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4bsg\" (UniqueName: \"kubernetes.io/projected/ddf405d5-caef-4e8f-8882-3d5a12a9d41e-kube-api-access-b4bsg\") pod \"maas-controller-65db4695c7-z9h72\" (UID: \"ddf405d5-caef-4e8f-8882-3d5a12a9d41e\") " pod="opendatahub/maas-controller-65db4695c7-z9h72"
Jul 28 14:33:12.655596 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:12.655452    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddf405d5-caef-4e8f-8882-3d5a12a9d41e-webhook-cert\") pod \"maas-controller-65db4695c7-z9h72\" (UID: \"ddf405d5-caef-4e8f-8882-3d5a12a9d41e\") " pod="opendatahub/maas-controller-65db4695c7-z9h72"
Jul 28 14:33:12.756587 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:12.756548    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4bsg\" (UniqueName: \"kubernetes.io/projected/ddf405d5-caef-4e8f-8882-3d5a12a9d41e-kube-api-access-b4bsg\") pod \"maas-controller-65db4695c7-z9h72\" (UID: \"ddf405d5-caef-4e8f-8882-3d5a12a9d41e\") " pod="opendatahub/maas-controller-65db4695c7-z9h72"
Jul 28 14:33:12.756805 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:12.756612    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddf405d5-caef-4e8f-8882-3d5a12a9d41e-webhook-cert\") pod \"maas-controller-65db4695c7-z9h72\" (UID: \"ddf405d5-caef-4e8f-8882-3d5a12a9d41e\") " pod="opendatahub/maas-controller-65db4695c7-z9h72"
Jul 28 14:33:12.758987 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:12.758961    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddf405d5-caef-4e8f-8882-3d5a12a9d41e-webhook-cert\") pod \"maas-controller-65db4695c7-z9h72\" (UID: \"ddf405d5-caef-4e8f-8882-3d5a12a9d41e\") " pod="opendatahub/maas-controller-65db4695c7-z9h72"
Jul 28 14:33:12.765513 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:12.765448    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4bsg\" (UniqueName: \"kubernetes.io/projected/ddf405d5-caef-4e8f-8882-3d5a12a9d41e-kube-api-access-b4bsg\") pod \"maas-controller-65db4695c7-z9h72\" (UID: \"ddf405d5-caef-4e8f-8882-3d5a12a9d41e\") " pod="opendatahub/maas-controller-65db4695c7-z9h72"
Jul 28 14:33:12.801226 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:12.801192    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1e281da-9804-49ea-b9c0-a67fd3edb09d" path="/var/lib/kubelet/pods/d1e281da-9804-49ea-b9c0-a67fd3edb09d/volumes"
Jul 28 14:33:12.830300 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:12.830260    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65db4695c7-z9h72"
Jul 28 14:33:12.968792 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:12.968767    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65db4695c7-z9h72"]
Jul 28 14:33:12.970952 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:33:12.970921    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddf405d5_caef_4e8f_8882_3d5a12a9d41e.slice/crio-cd8d4d1c1a4bf5309c8bb970c1befd625adbc7b7f407607c3da171017ccd7a83 WatchSource:0}: Error finding container cd8d4d1c1a4bf5309c8bb970c1befd625adbc7b7f407607c3da171017ccd7a83: Status 404 returned error can't find the container with id cd8d4d1c1a4bf5309c8bb970c1befd625adbc7b7f407607c3da171017ccd7a83
Jul 28 14:33:13.600845 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:13.600813    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65db4695c7-z9h72" event={"ID":"ddf405d5-caef-4e8f-8882-3d5a12a9d41e","Type":"ContainerStarted","Data":"79bae27b624fb314d6dc3390b175c5881f8e0848073213bfdbe6f69a71447756"}
Jul 28 14:33:13.601278 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:13.600864    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65db4695c7-z9h72" event={"ID":"ddf405d5-caef-4e8f-8882-3d5a12a9d41e","Type":"ContainerStarted","Data":"cd8d4d1c1a4bf5309c8bb970c1befd625adbc7b7f407607c3da171017ccd7a83"}
Jul 28 14:33:13.601278 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:13.600924    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65db4695c7-z9h72"
Jul 28 14:33:13.619591 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:13.619534    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65db4695c7-z9h72" podStartSLOduration=1.125895856 podStartE2EDuration="1.619515631s" totalImagesPullingTime="493.619775ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:33:13.618419699 +0000 UTC m=+863.419961916" watchObservedRunningTime="2026-07-28 14:33:13.619515631 +0000 UTC m=+863.421057845"
Jul 28 14:33:24.608440 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:33:24.608402    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65db4695c7-z9h72"
Jul 28 14:46:44.990988 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:44.990945    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"]
Jul 28 14:46:44.994495 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:44.994474    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"
Jul 28 14:46:44.996818 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:44.996797    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b99a3990-serving-cert\""
Jul 28 14:46:45.006155 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.006122    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"]
Jul 28 14:46:45.176231 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.176191    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cbdaa62-5c88-4c2a-a299-c6052c775dee-maas-api-tls\") pod \"maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r\" (UID: \"2cbdaa62-5c88-4c2a-a299-c6052c775dee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"
Jul 28 14:46:45.176418 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.176243    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snhlt\" (UniqueName: \"kubernetes.io/projected/2cbdaa62-5c88-4c2a-a299-c6052c775dee-kube-api-access-snhlt\") pod \"maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r\" (UID: \"2cbdaa62-5c88-4c2a-a299-c6052c775dee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"
Jul 28 14:46:45.176418 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.176385    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cbdaa62-5c88-4c2a-a299-c6052c775dee-openshift-service-ca\") pod \"maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r\" (UID: \"2cbdaa62-5c88-4c2a-a299-c6052c775dee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"
Jul 28 14:46:45.277572 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.277482    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cbdaa62-5c88-4c2a-a299-c6052c775dee-openshift-service-ca\") pod \"maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r\" (UID: \"2cbdaa62-5c88-4c2a-a299-c6052c775dee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"
Jul 28 14:46:45.277572 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.277536    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cbdaa62-5c88-4c2a-a299-c6052c775dee-maas-api-tls\") pod \"maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r\" (UID: \"2cbdaa62-5c88-4c2a-a299-c6052c775dee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"
Jul 28 14:46:45.277572 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.277561    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snhlt\" (UniqueName: \"kubernetes.io/projected/2cbdaa62-5c88-4c2a-a299-c6052c775dee-kube-api-access-snhlt\") pod \"maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r\" (UID: \"2cbdaa62-5c88-4c2a-a299-c6052c775dee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"
Jul 28 14:46:45.278079 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.278059    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cbdaa62-5c88-4c2a-a299-c6052c775dee-openshift-service-ca\") pod \"maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r\" (UID: \"2cbdaa62-5c88-4c2a-a299-c6052c775dee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"
Jul 28 14:46:45.279962 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.279939    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cbdaa62-5c88-4c2a-a299-c6052c775dee-maas-api-tls\") pod \"maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r\" (UID: \"2cbdaa62-5c88-4c2a-a299-c6052c775dee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"
Jul 28 14:46:45.286114 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.286074    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snhlt\" (UniqueName: \"kubernetes.io/projected/2cbdaa62-5c88-4c2a-a299-c6052c775dee-kube-api-access-snhlt\") pod \"maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r\" (UID: \"2cbdaa62-5c88-4c2a-a299-c6052c775dee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"
Jul 28 14:46:45.304865 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.304835    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"
Jul 28 14:46:45.444080 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.444048    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"]
Jul 28 14:46:45.446748 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:46:45.446712    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cbdaa62_5c88_4c2a_a299_c6052c775dee.slice/crio-3d5bce868fdada60f1d9c2a632cac13f51b24ed6f52b87f92d76c0e46acf8436 WatchSource:0}: Error finding container 3d5bce868fdada60f1d9c2a632cac13f51b24ed6f52b87f92d76c0e46acf8436: Status 404 returned error can't find the container with id 3d5bce868fdada60f1d9c2a632cac13f51b24ed6f52b87f92d76c0e46acf8436
Jul 28 14:46:45.448145 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.448127    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 14:46:45.670139 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.670101    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz"]
Jul 28 14:46:45.674389 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.674366    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz"
Jul 28 14:46:45.676717 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.676698    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-b99a3990\""
Jul 28 14:46:45.682691 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.682667    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz"]
Jul 28 14:46:45.704084 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.704047    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc"]
Jul 28 14:46:45.707187 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.707168    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc"
Jul 28 14:46:45.710425 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.710401    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-b99a3990-dockercfg-mn4ff\""
Jul 28 14:46:45.717147 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.717121    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc"]
Jul 28 14:46:45.781059 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.781020    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4880b6c1-3d47-4c78-acd3-c15f573d2081-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz\" (UID: \"4880b6c1-3d47-4c78-acd3-c15f573d2081\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz"
Jul 28 14:46:45.781256 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.781077    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nkt2\" (UniqueName: \"kubernetes.io/projected/4880b6c1-3d47-4c78-acd3-c15f573d2081-kube-api-access-6nkt2\") pod \"payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz\" (UID: \"4880b6c1-3d47-4c78-acd3-c15f573d2081\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz"
Jul 28 14:46:45.881949 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.881892    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7wfb\" (UniqueName: \"kubernetes.io/projected/f26edad7-302f-4845-8033-49d7c005925a-kube-api-access-j7wfb\") pod \"payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc\" (UID: \"f26edad7-302f-4845-8033-49d7c005925a\") " pod="openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc"
Jul 28 14:46:45.882150 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.881970    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f26edad7-302f-4845-8033-49d7c005925a-plugins-config-volume\") pod \"payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc\" (UID: \"f26edad7-302f-4845-8033-49d7c005925a\") " pod="openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc"
Jul 28 14:46:45.882150 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.881994    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4880b6c1-3d47-4c78-acd3-c15f573d2081-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz\" (UID: \"4880b6c1-3d47-4c78-acd3-c15f573d2081\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz"
Jul 28 14:46:45.882150 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.882017    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nkt2\" (UniqueName: \"kubernetes.io/projected/4880b6c1-3d47-4c78-acd3-c15f573d2081-kube-api-access-6nkt2\") pod \"payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz\" (UID: \"4880b6c1-3d47-4c78-acd3-c15f573d2081\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz"
Jul 28 14:46:45.882714 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.882694    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4880b6c1-3d47-4c78-acd3-c15f573d2081-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz\" (UID: \"4880b6c1-3d47-4c78-acd3-c15f573d2081\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz"
Jul 28 14:46:45.891935 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.891896    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nkt2\" (UniqueName: \"kubernetes.io/projected/4880b6c1-3d47-4c78-acd3-c15f573d2081-kube-api-access-6nkt2\") pod \"payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz\" (UID: \"4880b6c1-3d47-4c78-acd3-c15f573d2081\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz"
Jul 28 14:46:45.983018 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.982946    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f26edad7-302f-4845-8033-49d7c005925a-plugins-config-volume\") pod \"payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc\" (UID: \"f26edad7-302f-4845-8033-49d7c005925a\") " pod="openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc"
Jul 28 14:46:45.983188 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.983046    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7wfb\" (UniqueName: \"kubernetes.io/projected/f26edad7-302f-4845-8033-49d7c005925a-kube-api-access-j7wfb\") pod \"payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc\" (UID: \"f26edad7-302f-4845-8033-49d7c005925a\") " pod="openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc"
Jul 28 14:46:45.983652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.983631    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f26edad7-302f-4845-8033-49d7c005925a-plugins-config-volume\") pod \"payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc\" (UID: \"f26edad7-302f-4845-8033-49d7c005925a\") " pod="openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc"
Jul 28 14:46:45.985738 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.985714    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz"
Jul 28 14:46:45.991771 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:45.991740    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7wfb\" (UniqueName: \"kubernetes.io/projected/f26edad7-302f-4845-8033-49d7c005925a-kube-api-access-j7wfb\") pod \"payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc\" (UID: \"f26edad7-302f-4845-8033-49d7c005925a\") " pod="openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc"
Jul 28 14:46:46.018372 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:46.018334    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc"
Jul 28 14:46:46.148669 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:46.148623    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz"]
Jul 28 14:46:46.153368 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:46:46.153330    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4880b6c1_3d47_4c78_acd3_c15f573d2081.slice/crio-72769789c8b1731689e3014c92cab47242c3a27bb8583286a3094cc1f1c52dbe WatchSource:0}: Error finding container 72769789c8b1731689e3014c92cab47242c3a27bb8583286a3094cc1f1c52dbe: Status 404 returned error can't find the container with id 72769789c8b1731689e3014c92cab47242c3a27bb8583286a3094cc1f1c52dbe
Jul 28 14:46:46.186269 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:46.186236    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc"]
Jul 28 14:46:46.189369 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:46:46.189333    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf26edad7_302f_4845_8033_49d7c005925a.slice/crio-a59e20edc255a7f13750ce258a70ed718cecd1cdacbff0a1b1f73f146fd960a6 WatchSource:0}: Error finding container a59e20edc255a7f13750ce258a70ed718cecd1cdacbff0a1b1f73f146fd960a6: Status 404 returned error can't find the container with id a59e20edc255a7f13750ce258a70ed718cecd1cdacbff0a1b1f73f146fd960a6
Jul 28 14:46:46.357307 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:46.356895    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz" event={"ID":"4880b6c1-3d47-4c78-acd3-c15f573d2081","Type":"ContainerStarted","Data":"e9476e500400e0042b207a038317610e0e97a21677d2391c1f161c4f1bf32044"}
Jul 28 14:46:46.357307 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:46.356979    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz" event={"ID":"4880b6c1-3d47-4c78-acd3-c15f573d2081","Type":"ContainerStarted","Data":"72769789c8b1731689e3014c92cab47242c3a27bb8583286a3094cc1f1c52dbe"}
Jul 28 14:46:46.357307 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:46.357016    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz"
Jul 28 14:46:46.359057 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:46.359030    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc" event={"ID":"f26edad7-302f-4845-8033-49d7c005925a","Type":"ContainerStarted","Data":"ffe13a7b0f4a9de24a50b2f4015f0ec23a4eff2400056c2fe8c798d3e9b44aaf"}
Jul 28 14:46:46.359267 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:46.359077    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc" event={"ID":"f26edad7-302f-4845-8033-49d7c005925a","Type":"ContainerStarted","Data":"a59e20edc255a7f13750ce258a70ed718cecd1cdacbff0a1b1f73f146fd960a6"}
Jul 28 14:46:46.359267 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:46.359124    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc"
Jul 28 14:46:46.361003 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:46.360944    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r" event={"ID":"2cbdaa62-5c88-4c2a-a299-c6052c775dee","Type":"ContainerStarted","Data":"3a672eb0e736f930856f9e410008f81693a868ade1c3eb7914b250d3c0bad9a5"}
Jul 28 14:46:46.361144 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:46.361024    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r" event={"ID":"2cbdaa62-5c88-4c2a-a299-c6052c775dee","Type":"ContainerStarted","Data":"3d5bce868fdada60f1d9c2a632cac13f51b24ed6f52b87f92d76c0e46acf8436"}
Jul 28 14:46:46.361286 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:46.361269    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"
Jul 28 14:46:46.374861 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:46.374803    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz" podStartSLOduration=1.374782768 podStartE2EDuration="1.374782768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:46:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:46:46.373427824 +0000 UTC m=+1676.174970039" watchObservedRunningTime="2026-07-28 14:46:46.374782768 +0000 UTC m=+1676.176324987"
Jul 28 14:46:46.390048 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:46.389989    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc" podStartSLOduration=1.389970871 podStartE2EDuration="1.389970871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:46:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:46:46.388454142 +0000 UTC m=+1676.189996375" watchObservedRunningTime="2026-07-28 14:46:46.389970871 +0000 UTC m=+1676.191513085"
Jul 28 14:46:46.406504 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:46.406449    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r" podStartSLOduration=1.8646069490000001 podStartE2EDuration="2.406429655s" totalImagesPullingTime="541.822706ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:46:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:46:46.404551209 +0000 UTC m=+1676.206093437" watchObservedRunningTime="2026-07-28 14:46:46.406429655 +0000 UTC m=+1676.207971869"
Jul 28 14:46:52.371865 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:52.371827    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"
Jul 28 14:46:57.367861 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:57.367829    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc"
Jul 28 14:46:57.368310 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:46:57.368048    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz"
Jul 28 14:47:15.554912 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:15.554820    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"]
Jul 28 14:47:15.555421 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:15.555062    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r" podUID="2cbdaa62-5c88-4c2a-a299-c6052c775dee" containerName="maas-api" containerID="cri-o://3a672eb0e736f930856f9e410008f81693a868ade1c3eb7914b250d3c0bad9a5" gracePeriod=30
Jul 28 14:47:15.613674 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:15.613639    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc"]
Jul 28 14:47:15.613868 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:15.613838    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc" podUID="f26edad7-302f-4845-8033-49d7c005925a" containerName="payload-processing" containerID="cri-o://ffe13a7b0f4a9de24a50b2f4015f0ec23a4eff2400056c2fe8c798d3e9b44aaf" gracePeriod=30
Jul 28 14:47:15.619744 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:15.619703    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz"]
Jul 28 14:47:15.619992 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:15.619939    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz" podUID="4880b6c1-3d47-4c78-acd3-c15f573d2081" containerName="payload-pre-processing" containerID="cri-o://e9476e500400e0042b207a038317610e0e97a21677d2391c1f161c4f1bf32044" gracePeriod=30
Jul 28 14:47:15.926782 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:15.926756    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc"
Jul 28 14:47:15.928706 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:15.928683    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz"
Jul 28 14:47:15.942767 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:15.942744    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"
Jul 28 14:47:16.055567 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.055532    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cbdaa62-5c88-4c2a-a299-c6052c775dee-maas-api-tls\") pod \"2cbdaa62-5c88-4c2a-a299-c6052c775dee\" (UID: \"2cbdaa62-5c88-4c2a-a299-c6052c775dee\") "
Jul 28 14:47:16.055765 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.055589    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4880b6c1-3d47-4c78-acd3-c15f573d2081-plugins-config-volume\") pod \"4880b6c1-3d47-4c78-acd3-c15f573d2081\" (UID: \"4880b6c1-3d47-4c78-acd3-c15f573d2081\") "
Jul 28 14:47:16.055765 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.055618    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nkt2\" (UniqueName: \"kubernetes.io/projected/4880b6c1-3d47-4c78-acd3-c15f573d2081-kube-api-access-6nkt2\") pod \"4880b6c1-3d47-4c78-acd3-c15f573d2081\" (UID: \"4880b6c1-3d47-4c78-acd3-c15f573d2081\") "
Jul 28 14:47:16.055765 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.055641    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7wfb\" (UniqueName: \"kubernetes.io/projected/f26edad7-302f-4845-8033-49d7c005925a-kube-api-access-j7wfb\") pod \"f26edad7-302f-4845-8033-49d7c005925a\" (UID: \"f26edad7-302f-4845-8033-49d7c005925a\") "
Jul 28 14:47:16.055765 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.055725    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snhlt\" (UniqueName: \"kubernetes.io/projected/2cbdaa62-5c88-4c2a-a299-c6052c775dee-kube-api-access-snhlt\") pod \"2cbdaa62-5c88-4c2a-a299-c6052c775dee\" (UID: \"2cbdaa62-5c88-4c2a-a299-c6052c775dee\") "
Jul 28 14:47:16.055765 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.055760    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f26edad7-302f-4845-8033-49d7c005925a-plugins-config-volume\") pod \"f26edad7-302f-4845-8033-49d7c005925a\" (UID: \"f26edad7-302f-4845-8033-49d7c005925a\") "
Jul 28 14:47:16.056020 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.055811    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cbdaa62-5c88-4c2a-a299-c6052c775dee-openshift-service-ca\") pod \"2cbdaa62-5c88-4c2a-a299-c6052c775dee\" (UID: \"2cbdaa62-5c88-4c2a-a299-c6052c775dee\") "
Jul 28 14:47:16.056237 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.056204    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f26edad7-302f-4845-8033-49d7c005925a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f26edad7-302f-4845-8033-49d7c005925a" (UID: "f26edad7-302f-4845-8033-49d7c005925a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:47:16.056349 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.056209    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4880b6c1-3d47-4c78-acd3-c15f573d2081-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4880b6c1-3d47-4c78-acd3-c15f573d2081" (UID: "4880b6c1-3d47-4c78-acd3-c15f573d2081"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:47:16.056469 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.056450    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cbdaa62-5c88-4c2a-a299-c6052c775dee-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2cbdaa62-5c88-4c2a-a299-c6052c775dee" (UID: "2cbdaa62-5c88-4c2a-a299-c6052c775dee"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:47:16.057852 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.057823    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f26edad7-302f-4845-8033-49d7c005925a-kube-api-access-j7wfb" (OuterVolumeSpecName: "kube-api-access-j7wfb") pod "f26edad7-302f-4845-8033-49d7c005925a" (UID: "f26edad7-302f-4845-8033-49d7c005925a"). InnerVolumeSpecName "kube-api-access-j7wfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:47:16.057953 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.057845    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cbdaa62-5c88-4c2a-a299-c6052c775dee-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2cbdaa62-5c88-4c2a-a299-c6052c775dee" (UID: "2cbdaa62-5c88-4c2a-a299-c6052c775dee"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:47:16.058158 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.058134    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cbdaa62-5c88-4c2a-a299-c6052c775dee-kube-api-access-snhlt" (OuterVolumeSpecName: "kube-api-access-snhlt") pod "2cbdaa62-5c88-4c2a-a299-c6052c775dee" (UID: "2cbdaa62-5c88-4c2a-a299-c6052c775dee"). InnerVolumeSpecName "kube-api-access-snhlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:47:16.058243 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.058136    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4880b6c1-3d47-4c78-acd3-c15f573d2081-kube-api-access-6nkt2" (OuterVolumeSpecName: "kube-api-access-6nkt2") pod "4880b6c1-3d47-4c78-acd3-c15f573d2081" (UID: "4880b6c1-3d47-4c78-acd3-c15f573d2081"). InnerVolumeSpecName "kube-api-access-6nkt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:47:16.156741 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.156706    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snhlt\" (UniqueName: \"kubernetes.io/projected/2cbdaa62-5c88-4c2a-a299-c6052c775dee-kube-api-access-snhlt\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:16.156741 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.156736    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f26edad7-302f-4845-8033-49d7c005925a-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:16.156741 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.156750    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cbdaa62-5c88-4c2a-a299-c6052c775dee-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:16.156978 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.156763    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cbdaa62-5c88-4c2a-a299-c6052c775dee-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:16.156978 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.156774    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4880b6c1-3d47-4c78-acd3-c15f573d2081-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:16.156978 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.156790    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nkt2\" (UniqueName: \"kubernetes.io/projected/4880b6c1-3d47-4c78-acd3-c15f573d2081-kube-api-access-6nkt2\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:16.156978 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.156800    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7wfb\" (UniqueName: \"kubernetes.io/projected/f26edad7-302f-4845-8033-49d7c005925a-kube-api-access-j7wfb\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:16.508066 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.507976    2584 generic.go:356] "Generic (PLEG): container finished" podID="f26edad7-302f-4845-8033-49d7c005925a" containerID="ffe13a7b0f4a9de24a50b2f4015f0ec23a4eff2400056c2fe8c798d3e9b44aaf" exitCode=0
Jul 28 14:47:16.508066 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.508025    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc"
Jul 28 14:47:16.508066 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.508057    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc" event={"ID":"f26edad7-302f-4845-8033-49d7c005925a","Type":"ContainerDied","Data":"ffe13a7b0f4a9de24a50b2f4015f0ec23a4eff2400056c2fe8c798d3e9b44aaf"}
Jul 28 14:47:16.508376 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.508134    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc" event={"ID":"f26edad7-302f-4845-8033-49d7c005925a","Type":"ContainerDied","Data":"a59e20edc255a7f13750ce258a70ed718cecd1cdacbff0a1b1f73f146fd960a6"}
Jul 28 14:47:16.508376 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.508166    2584 scope.go:117] "RemoveContainer" containerID="ffe13a7b0f4a9de24a50b2f4015f0ec23a4eff2400056c2fe8c798d3e9b44aaf"
Jul 28 14:47:16.509397 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.509378    2584 generic.go:356] "Generic (PLEG): container finished" podID="2cbdaa62-5c88-4c2a-a299-c6052c775dee" containerID="3a672eb0e736f930856f9e410008f81693a868ade1c3eb7914b250d3c0bad9a5" exitCode=0
Jul 28 14:47:16.509499 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.509440    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"
Jul 28 14:47:16.509557 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.509511    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r" event={"ID":"2cbdaa62-5c88-4c2a-a299-c6052c775dee","Type":"ContainerDied","Data":"3a672eb0e736f930856f9e410008f81693a868ade1c3eb7914b250d3c0bad9a5"}
Jul 28 14:47:16.509610 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.509562    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r" event={"ID":"2cbdaa62-5c88-4c2a-a299-c6052c775dee","Type":"ContainerDied","Data":"3d5bce868fdada60f1d9c2a632cac13f51b24ed6f52b87f92d76c0e46acf8436"}
Jul 28 14:47:16.510787 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.510765    2584 generic.go:356] "Generic (PLEG): container finished" podID="4880b6c1-3d47-4c78-acd3-c15f573d2081" containerID="e9476e500400e0042b207a038317610e0e97a21677d2391c1f161c4f1bf32044" exitCode=0
Jul 28 14:47:16.510902 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.510795    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz" event={"ID":"4880b6c1-3d47-4c78-acd3-c15f573d2081","Type":"ContainerDied","Data":"e9476e500400e0042b207a038317610e0e97a21677d2391c1f161c4f1bf32044"}
Jul 28 14:47:16.510902 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.510815    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz"
Jul 28 14:47:16.510902 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.510856    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz" event={"ID":"4880b6c1-3d47-4c78-acd3-c15f573d2081","Type":"ContainerDied","Data":"72769789c8b1731689e3014c92cab47242c3a27bb8583286a3094cc1f1c52dbe"}
Jul 28 14:47:16.519924 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.519876    2584 scope.go:117] "RemoveContainer" containerID="ffe13a7b0f4a9de24a50b2f4015f0ec23a4eff2400056c2fe8c798d3e9b44aaf"
Jul 28 14:47:16.520671 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:47:16.520646    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffe13a7b0f4a9de24a50b2f4015f0ec23a4eff2400056c2fe8c798d3e9b44aaf\": container with ID starting with ffe13a7b0f4a9de24a50b2f4015f0ec23a4eff2400056c2fe8c798d3e9b44aaf not found: ID does not exist" containerID="ffe13a7b0f4a9de24a50b2f4015f0ec23a4eff2400056c2fe8c798d3e9b44aaf"
Jul 28 14:47:16.520777 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.520691    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffe13a7b0f4a9de24a50b2f4015f0ec23a4eff2400056c2fe8c798d3e9b44aaf"} err="failed to get container status \"ffe13a7b0f4a9de24a50b2f4015f0ec23a4eff2400056c2fe8c798d3e9b44aaf\": rpc error: code = NotFound desc = could not find container \"ffe13a7b0f4a9de24a50b2f4015f0ec23a4eff2400056c2fe8c798d3e9b44aaf\": container with ID starting with ffe13a7b0f4a9de24a50b2f4015f0ec23a4eff2400056c2fe8c798d3e9b44aaf not found: ID does not exist"
Jul 28 14:47:16.520842 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.520783    2584 scope.go:117] "RemoveContainer" containerID="3a672eb0e736f930856f9e410008f81693a868ade1c3eb7914b250d3c0bad9a5"
Jul 28 14:47:16.531142 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.531127    2584 scope.go:117] "RemoveContainer" containerID="3a672eb0e736f930856f9e410008f81693a868ade1c3eb7914b250d3c0bad9a5"
Jul 28 14:47:16.531380 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:47:16.531362    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a672eb0e736f930856f9e410008f81693a868ade1c3eb7914b250d3c0bad9a5\": container with ID starting with 3a672eb0e736f930856f9e410008f81693a868ade1c3eb7914b250d3c0bad9a5 not found: ID does not exist" containerID="3a672eb0e736f930856f9e410008f81693a868ade1c3eb7914b250d3c0bad9a5"
Jul 28 14:47:16.531419 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.531387    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a672eb0e736f930856f9e410008f81693a868ade1c3eb7914b250d3c0bad9a5"} err="failed to get container status \"3a672eb0e736f930856f9e410008f81693a868ade1c3eb7914b250d3c0bad9a5\": rpc error: code = NotFound desc = could not find container \"3a672eb0e736f930856f9e410008f81693a868ade1c3eb7914b250d3c0bad9a5\": container with ID starting with 3a672eb0e736f930856f9e410008f81693a868ade1c3eb7914b250d3c0bad9a5 not found: ID does not exist"
Jul 28 14:47:16.531419 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.531403    2584 scope.go:117] "RemoveContainer" containerID="e9476e500400e0042b207a038317610e0e97a21677d2391c1f161c4f1bf32044"
Jul 28 14:47:16.539268 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.539247    2584 scope.go:117] "RemoveContainer" containerID="e9476e500400e0042b207a038317610e0e97a21677d2391c1f161c4f1bf32044"
Jul 28 14:47:16.539540 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:47:16.539523    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9476e500400e0042b207a038317610e0e97a21677d2391c1f161c4f1bf32044\": container with ID starting with e9476e500400e0042b207a038317610e0e97a21677d2391c1f161c4f1bf32044 not found: ID does not exist" containerID="e9476e500400e0042b207a038317610e0e97a21677d2391c1f161c4f1bf32044"
Jul 28 14:47:16.539591 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.539547    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9476e500400e0042b207a038317610e0e97a21677d2391c1f161c4f1bf32044"} err="failed to get container status \"e9476e500400e0042b207a038317610e0e97a21677d2391c1f161c4f1bf32044\": rpc error: code = NotFound desc = could not find container \"e9476e500400e0042b207a038317610e0e97a21677d2391c1f161c4f1bf32044\": container with ID starting with e9476e500400e0042b207a038317610e0e97a21677d2391c1f161c4f1bf32044 not found: ID does not exist"
Jul 28 14:47:16.539883 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.539865    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc"]
Jul 28 14:47:16.545257 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.545235    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b99a3990-fd9b9489b-6k7zc"]
Jul 28 14:47:16.558682 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.558659    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"]
Jul 28 14:47:16.561749 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.561726    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b99a3990-7bd9698b8-lzr8r"]
Jul 28 14:47:16.575048 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.575023    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz"]
Jul 28 14:47:16.577472 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.577453    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b99a3990-fff9b65b-2ckjz"]
Jul 28 14:47:16.796848 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.796772    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cbdaa62-5c88-4c2a-a299-c6052c775dee" path="/var/lib/kubelet/pods/2cbdaa62-5c88-4c2a-a299-c6052c775dee/volumes"
Jul 28 14:47:16.797183 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.797169    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4880b6c1-3d47-4c78-acd3-c15f573d2081" path="/var/lib/kubelet/pods/4880b6c1-3d47-4c78-acd3-c15f573d2081/volumes"
Jul 28 14:47:16.797501 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:16.797489    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f26edad7-302f-4845-8033-49d7c005925a" path="/var/lib/kubelet/pods/f26edad7-302f-4845-8033-49d7c005925a/volumes"
Jul 28 14:47:17.045806 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.045773    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"]
Jul 28 14:47:17.046160 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.046147    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cbdaa62-5c88-4c2a-a299-c6052c775dee" containerName="maas-api"
Jul 28 14:47:17.046220 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.046162    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cbdaa62-5c88-4c2a-a299-c6052c775dee" containerName="maas-api"
Jul 28 14:47:17.046220 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.046171    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f26edad7-302f-4845-8033-49d7c005925a" containerName="payload-processing"
Jul 28 14:47:17.046220 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.046176    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f26edad7-302f-4845-8033-49d7c005925a" containerName="payload-processing"
Jul 28 14:47:17.046220 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.046194    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4880b6c1-3d47-4c78-acd3-c15f573d2081" containerName="payload-pre-processing"
Jul 28 14:47:17.046220 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.046200    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4880b6c1-3d47-4c78-acd3-c15f573d2081" containerName="payload-pre-processing"
Jul 28 14:47:17.046395 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.046257    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4880b6c1-3d47-4c78-acd3-c15f573d2081" containerName="payload-pre-processing"
Jul 28 14:47:17.046395 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.046264    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f26edad7-302f-4845-8033-49d7c005925a" containerName="payload-processing"
Jul 28 14:47:17.046395 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.046270    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cbdaa62-5c88-4c2a-a299-c6052c775dee" containerName="maas-api"
Jul 28 14:47:17.050528 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.050469    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.053598 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.053578    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-35b9d8ff-gw-openshift-default-dockercfg-llss4\""
Jul 28 14:47:17.066369 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.066341    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"]
Jul 28 14:47:17.165951 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.165916    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-workload-socket\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.165951 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.165954    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ea36761-2127-46f2-8362-7420e680d39c-istiod-ca-cert\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.166173 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.165985    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-workload-certs\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.166173 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.166007    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-istio-envoy\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.166173 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.166070    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-credential-socket\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.166173 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.166133    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-istio-data\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.166173 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.166153    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ea36761-2127-46f2-8362-7420e680d39c-istio-podinfo\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.166339 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.166222    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zjtd\" (UniqueName: \"kubernetes.io/projected/7ea36761-2127-46f2-8362-7420e680d39c-kube-api-access-7zjtd\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.166339 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.166262    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ea36761-2127-46f2-8362-7420e680d39c-istio-token\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.267379 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.267336    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-workload-socket\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.267379 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.267380    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ea36761-2127-46f2-8362-7420e680d39c-istiod-ca-cert\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.267637 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.267400    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-workload-certs\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.267637 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.267448    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-istio-envoy\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.267637 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.267493    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-credential-socket\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.267637 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.267530    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-istio-data\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.267637 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.267552    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ea36761-2127-46f2-8362-7420e680d39c-istio-podinfo\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.267637 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.267620    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zjtd\" (UniqueName: \"kubernetes.io/projected/7ea36761-2127-46f2-8362-7420e680d39c-kube-api-access-7zjtd\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.267922 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.267672    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ea36761-2127-46f2-8362-7420e680d39c-istio-token\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.267922 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.267882    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-workload-certs\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.268028 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.267938    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-istio-data\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.268196 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.268171    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-credential-socket\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.268292 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.268185    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-workload-socket\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.268292 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.268245    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ea36761-2127-46f2-8362-7420e680d39c-istiod-ca-cert\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.269849 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.269832    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-istio-envoy\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.270081 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.270058    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ea36761-2127-46f2-8362-7420e680d39c-istio-podinfo\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.277713 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.277692    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ea36761-2127-46f2-8362-7420e680d39c-istio-token\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.277960 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.277942    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zjtd\" (UniqueName: \"kubernetes.io/projected/7ea36761-2127-46f2-8362-7420e680d39c-kube-api-access-7zjtd\") pod \"e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") " pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.360785 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.360750    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:17.504627 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.504456    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"]
Jul 28 14:47:17.507275 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:47:17.507239    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ea36761_2127_46f2_8362_7420e680d39c.slice/crio-5dc7dec006e5f4691168f337ca66fa0624374b4b9f9c684ce515220f32d89995 WatchSource:0}: Error finding container 5dc7dec006e5f4691168f337ca66fa0624374b4b9f9c684ce515220f32d89995: Status 404 returned error can't find the container with id 5dc7dec006e5f4691168f337ca66fa0624374b4b9f9c684ce515220f32d89995
Jul 28 14:47:17.509272 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.509242    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:47:17.509368 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.509305    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:47:17.509368 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.509335    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:47:17.517047 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:17.517020    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv" event={"ID":"7ea36761-2127-46f2-8362-7420e680d39c","Type":"ContainerStarted","Data":"5dc7dec006e5f4691168f337ca66fa0624374b4b9f9c684ce515220f32d89995"}
Jul 28 14:47:18.525447 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:18.525412    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv" event={"ID":"7ea36761-2127-46f2-8362-7420e680d39c","Type":"ContainerStarted","Data":"77447163aba9a82ab04719f331d94aee9a9198476f48e858d94c95fba748696a"}
Jul 28 14:47:18.554680 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:18.554622    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv" podStartSLOduration=1.554603942 podStartE2EDuration="1.554603942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:47:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:47:18.552627141 +0000 UTC m=+1708.354169355" watchObservedRunningTime="2026-07-28 14:47:18.554603942 +0000 UTC m=+1708.356146156"
Jul 28 14:47:19.361611 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:19.361570    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:19.366412 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:19.366384    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:19.529577 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:19.529545    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:19.530754 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:19.530733    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:20.239468 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.239437    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"]
Jul 28 14:47:20.243001 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.242983    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"
Jul 28 14:47:20.246187 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.246167    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-35b9d8ff-serving-cert\""
Jul 28 14:47:20.261360 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.261327    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"]
Jul 28 14:47:20.397012 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.396977    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8bl9\" (UniqueName: \"kubernetes.io/projected/8bc0fd80-e941-4202-88f4-0f21f0ca35a4-kube-api-access-m8bl9\") pod \"maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287\" (UID: \"8bc0fd80-e941-4202-88f4-0f21f0ca35a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"
Jul 28 14:47:20.397238 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.397032    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8bc0fd80-e941-4202-88f4-0f21f0ca35a4-maas-api-tls\") pod \"maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287\" (UID: \"8bc0fd80-e941-4202-88f4-0f21f0ca35a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"
Jul 28 14:47:20.397238 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.397167    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bc0fd80-e941-4202-88f4-0f21f0ca35a4-openshift-service-ca\") pod \"maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287\" (UID: \"8bc0fd80-e941-4202-88f4-0f21f0ca35a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"
Jul 28 14:47:20.498601 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.498507    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bc0fd80-e941-4202-88f4-0f21f0ca35a4-openshift-service-ca\") pod \"maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287\" (UID: \"8bc0fd80-e941-4202-88f4-0f21f0ca35a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"
Jul 28 14:47:20.498601 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.498560    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8bl9\" (UniqueName: \"kubernetes.io/projected/8bc0fd80-e941-4202-88f4-0f21f0ca35a4-kube-api-access-m8bl9\") pod \"maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287\" (UID: \"8bc0fd80-e941-4202-88f4-0f21f0ca35a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"
Jul 28 14:47:20.498791 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.498604    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8bc0fd80-e941-4202-88f4-0f21f0ca35a4-maas-api-tls\") pod \"maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287\" (UID: \"8bc0fd80-e941-4202-88f4-0f21f0ca35a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"
Jul 28 14:47:20.499195 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.499172    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bc0fd80-e941-4202-88f4-0f21f0ca35a4-openshift-service-ca\") pod \"maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287\" (UID: \"8bc0fd80-e941-4202-88f4-0f21f0ca35a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"
Jul 28 14:47:20.501018 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.500996    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8bc0fd80-e941-4202-88f4-0f21f0ca35a4-maas-api-tls\") pod \"maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287\" (UID: \"8bc0fd80-e941-4202-88f4-0f21f0ca35a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"
Jul 28 14:47:20.507356 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.507328    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8bl9\" (UniqueName: \"kubernetes.io/projected/8bc0fd80-e941-4202-88f4-0f21f0ca35a4-kube-api-access-m8bl9\") pod \"maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287\" (UID: \"8bc0fd80-e941-4202-88f4-0f21f0ca35a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"
Jul 28 14:47:20.552870 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.552835    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"
Jul 28 14:47:20.704551 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.704525    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"]
Jul 28 14:47:20.706525 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:47:20.706502    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bc0fd80_e941_4202_88f4_0f21f0ca35a4.slice/crio-a4c725afddb33203c4ee79a1da2b3536541934424a0d499b49debe495a70a88e WatchSource:0}: Error finding container a4c725afddb33203c4ee79a1da2b3536541934424a0d499b49debe495a70a88e: Status 404 returned error can't find the container with id a4c725afddb33203c4ee79a1da2b3536541934424a0d499b49debe495a70a88e
Jul 28 14:47:20.937856 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.937826    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq"]
Jul 28 14:47:20.942446 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.942429    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq"
Jul 28 14:47:20.945217 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.945188    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-35b9d8ff-dockercfg-nph6t\""
Jul 28 14:47:20.945217 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.945210    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-35b9d8ff\""
Jul 28 14:47:20.954619 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.954595    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq"]
Jul 28 14:47:20.986395 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.986355    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg"]
Jul 28 14:47:20.989734 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:20.989713    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg"
Jul 28 14:47:21.000790 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.000367    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg"]
Jul 28 14:47:21.105324 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.105294    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj28f\" (UniqueName: \"kubernetes.io/projected/a35f8981-c633-4bea-9e5f-ec0f53daf81d-kube-api-access-hj28f\") pod \"payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg\" (UID: \"a35f8981-c633-4bea-9e5f-ec0f53daf81d\") " pod="openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg"
Jul 28 14:47:21.105492 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.105390    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49ce74c8-f4fb-4a06-953e-7f23354d9e24-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq\" (UID: \"49ce74c8-f4fb-4a06-953e-7f23354d9e24\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq"
Jul 28 14:47:21.105492 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.105455    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a35f8981-c633-4bea-9e5f-ec0f53daf81d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg\" (UID: \"a35f8981-c633-4bea-9e5f-ec0f53daf81d\") " pod="openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg"
Jul 28 14:47:21.105492 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.105477    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkrc7\" (UniqueName: \"kubernetes.io/projected/49ce74c8-f4fb-4a06-953e-7f23354d9e24-kube-api-access-gkrc7\") pod \"payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq\" (UID: \"49ce74c8-f4fb-4a06-953e-7f23354d9e24\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq"
Jul 28 14:47:21.205939 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.205910    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a35f8981-c633-4bea-9e5f-ec0f53daf81d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg\" (UID: \"a35f8981-c633-4bea-9e5f-ec0f53daf81d\") " pod="openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg"
Jul 28 14:47:21.206052 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.205948    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkrc7\" (UniqueName: \"kubernetes.io/projected/49ce74c8-f4fb-4a06-953e-7f23354d9e24-kube-api-access-gkrc7\") pod \"payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq\" (UID: \"49ce74c8-f4fb-4a06-953e-7f23354d9e24\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq"
Jul 28 14:47:21.206052 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.206004    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj28f\" (UniqueName: \"kubernetes.io/projected/a35f8981-c633-4bea-9e5f-ec0f53daf81d-kube-api-access-hj28f\") pod \"payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg\" (UID: \"a35f8981-c633-4bea-9e5f-ec0f53daf81d\") " pod="openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg"
Jul 28 14:47:21.206202 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.206067    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49ce74c8-f4fb-4a06-953e-7f23354d9e24-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq\" (UID: \"49ce74c8-f4fb-4a06-953e-7f23354d9e24\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq"
Jul 28 14:47:21.206695 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.206672    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a35f8981-c633-4bea-9e5f-ec0f53daf81d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg\" (UID: \"a35f8981-c633-4bea-9e5f-ec0f53daf81d\") " pod="openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg"
Jul 28 14:47:21.206805 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.206787    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49ce74c8-f4fb-4a06-953e-7f23354d9e24-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq\" (UID: \"49ce74c8-f4fb-4a06-953e-7f23354d9e24\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq"
Jul 28 14:47:21.214424 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.214396    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj28f\" (UniqueName: \"kubernetes.io/projected/a35f8981-c633-4bea-9e5f-ec0f53daf81d-kube-api-access-hj28f\") pod \"payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg\" (UID: \"a35f8981-c633-4bea-9e5f-ec0f53daf81d\") " pod="openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg"
Jul 28 14:47:21.214512 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.214468    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkrc7\" (UniqueName: \"kubernetes.io/projected/49ce74c8-f4fb-4a06-953e-7f23354d9e24-kube-api-access-gkrc7\") pod \"payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq\" (UID: \"49ce74c8-f4fb-4a06-953e-7f23354d9e24\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq"
Jul 28 14:47:21.253730 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.253709    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq"
Jul 28 14:47:21.300955 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.300918    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg"
Jul 28 14:47:21.400365 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.400209    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq"]
Jul 28 14:47:21.404568 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:47:21.404539    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49ce74c8_f4fb_4a06_953e_7f23354d9e24.slice/crio-d82bf8869c644aea0c905a4e1c222371deef1406c407d4d4dadd799a907c27d6 WatchSource:0}: Error finding container d82bf8869c644aea0c905a4e1c222371deef1406c407d4d4dadd799a907c27d6: Status 404 returned error can't find the container with id d82bf8869c644aea0c905a4e1c222371deef1406c407d4d4dadd799a907c27d6
Jul 28 14:47:21.454982 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.454912    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg"]
Jul 28 14:47:21.458836 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:47:21.458810    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda35f8981_c633_4bea_9e5f_ec0f53daf81d.slice/crio-c441d70a6b866cf392b8b2b8b95e200d29be9842d7841b831c26fb068cd6059c WatchSource:0}: Error finding container c441d70a6b866cf392b8b2b8b95e200d29be9842d7841b831c26fb068cd6059c: Status 404 returned error can't find the container with id c441d70a6b866cf392b8b2b8b95e200d29be9842d7841b831c26fb068cd6059c
Jul 28 14:47:21.541466 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.541423    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287" event={"ID":"8bc0fd80-e941-4202-88f4-0f21f0ca35a4","Type":"ContainerStarted","Data":"7b0841c51d3d6891e3482f8e66041e923f12c5bfe4d187e422bf7da8d6c0e35f"}
Jul 28 14:47:21.541679 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.541511    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287" event={"ID":"8bc0fd80-e941-4202-88f4-0f21f0ca35a4","Type":"ContainerStarted","Data":"a4c725afddb33203c4ee79a1da2b3536541934424a0d499b49debe495a70a88e"}
Jul 28 14:47:21.541767 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.541682    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"
Jul 28 14:47:21.543374 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.543292    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg" event={"ID":"a35f8981-c633-4bea-9e5f-ec0f53daf81d","Type":"ContainerStarted","Data":"c4eb8ac70e99401f1c9fd225fad77bd40ba8546804f989f814ef8c592932aaaa"}
Jul 28 14:47:21.543374 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.543328    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg" event={"ID":"a35f8981-c633-4bea-9e5f-ec0f53daf81d","Type":"ContainerStarted","Data":"c441d70a6b866cf392b8b2b8b95e200d29be9842d7841b831c26fb068cd6059c"}
Jul 28 14:47:21.543374 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.543358    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg"
Jul 28 14:47:21.545062 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.545033    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq" event={"ID":"49ce74c8-f4fb-4a06-953e-7f23354d9e24","Type":"ContainerStarted","Data":"6a5b496821558b8cdc1f4f33f1054af5cef3791308c9c4aa00053d38b5f3870f"}
Jul 28 14:47:21.545200 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.545082    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq" event={"ID":"49ce74c8-f4fb-4a06-953e-7f23354d9e24","Type":"ContainerStarted","Data":"d82bf8869c644aea0c905a4e1c222371deef1406c407d4d4dadd799a907c27d6"}
Jul 28 14:47:21.545200 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.545137    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq"
Jul 28 14:47:21.558171 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.558087    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287" podStartSLOduration=1.080288023 podStartE2EDuration="1.558065226s" totalImagesPullingTime="477.777203ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:47:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:47:21.557667555 +0000 UTC m=+1711.359209772" watchObservedRunningTime="2026-07-28 14:47:21.558065226 +0000 UTC m=+1711.359607513"
Jul 28 14:47:21.575571 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.575491    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq" podStartSLOduration=1.5754687459999999 podStartE2EDuration="1.575468746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:47:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:47:21.575167767 +0000 UTC m=+1711.376709985" watchObservedRunningTime="2026-07-28 14:47:21.575468746 +0000 UTC m=+1711.377010962"
Jul 28 14:47:21.591442 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:21.591386    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg" podStartSLOduration=1.591368857 podStartE2EDuration="1.591368857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:47:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:47:21.590301683 +0000 UTC m=+1711.391843899" watchObservedRunningTime="2026-07-28 14:47:21.591368857 +0000 UTC m=+1711.392911072"
Jul 28 14:47:27.557710 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:27.557682    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"
Jul 28 14:47:32.553268 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:32.553236    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg"
Jul 28 14:47:32.553747 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:32.553392    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq"
Jul 28 14:47:33.611940 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:33.611906    2584 generic.go:356] "Generic (PLEG): container finished" podID="7ea36761-2127-46f2-8362-7420e680d39c" containerID="77447163aba9a82ab04719f331d94aee9a9198476f48e858d94c95fba748696a" exitCode=0
Jul 28 14:47:33.612683 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:33.611977    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv" event={"ID":"7ea36761-2127-46f2-8362-7420e680d39c","Type":"ContainerDied","Data":"77447163aba9a82ab04719f331d94aee9a9198476f48e858d94c95fba748696a"}
Jul 28 14:47:33.612683 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:33.612422    2584 scope.go:117] "RemoveContainer" containerID="77447163aba9a82ab04719f331d94aee9a9198476f48e858d94c95fba748696a"
Jul 28 14:47:33.613595 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:33.613561    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:47:33.613672 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:33.613655    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:47:33.613721 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:33.613707    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:47:34.530038 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:34.529987    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:34.619449 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:34.619417    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv" event={"ID":"7ea36761-2127-46f2-8362-7420e680d39c","Type":"ContainerStarted","Data":"8091fdea9d218983a93ee81bafd0f62771e0e4f83eb15e47075f3a47570fe287"}
Jul 28 14:47:35.361106 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:35.361050    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:35.362238 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:35.362211    2584 patch_prober.go:28] interesting pod/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 28 14:47:35.362385 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:35.362259    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv" podUID="7ea36761-2127-46f2-8362-7420e680d39c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 28 14:47:36.362042 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:36.362007    2584 patch_prober.go:28] interesting pod/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 28 14:47:36.362427 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:36.362127    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv" podUID="7ea36761-2127-46f2-8362-7420e680d39c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 28 14:47:37.361813 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:37.361776    2584 patch_prober.go:28] interesting pod/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 28 14:47:37.361993 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:37.361883    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv" podUID="7ea36761-2127-46f2-8362-7420e680d39c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 28 14:47:38.362231 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:38.362185    2584 patch_prober.go:28] interesting pod/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 28 14:47:38.362750 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:38.362246    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv" podUID="7ea36761-2127-46f2-8362-7420e680d39c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 28 14:47:39.365626 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:39.365593    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:39.366045 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:39.365889    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:39.366548 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:39.366528    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:51.506845 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.506808    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"]
Jul 28 14:47:51.507321 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.507027    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287" podUID="8bc0fd80-e941-4202-88f4-0f21f0ca35a4" containerName="maas-api" containerID="cri-o://7b0841c51d3d6891e3482f8e66041e923f12c5bfe4d187e422bf7da8d6c0e35f" gracePeriod=30
Jul 28 14:47:51.575708 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.575159    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg"]
Jul 28 14:47:51.576347 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.575945    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg" podUID="a35f8981-c633-4bea-9e5f-ec0f53daf81d" containerName="payload-processing" containerID="cri-o://c4eb8ac70e99401f1c9fd225fad77bd40ba8546804f989f814ef8c592932aaaa" gracePeriod=30
Jul 28 14:47:51.577774 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.577750    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq"]
Jul 28 14:47:51.578218 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.578146    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq" podUID="49ce74c8-f4fb-4a06-953e-7f23354d9e24" containerName="payload-pre-processing" containerID="cri-o://6a5b496821558b8cdc1f4f33f1054af5cef3791308c9c4aa00053d38b5f3870f" gracePeriod=30
Jul 28 14:47:51.708148 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.708081    2584 generic.go:356] "Generic (PLEG): container finished" podID="8bc0fd80-e941-4202-88f4-0f21f0ca35a4" containerID="7b0841c51d3d6891e3482f8e66041e923f12c5bfe4d187e422bf7da8d6c0e35f" exitCode=0
Jul 28 14:47:51.708148 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.708125    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287" event={"ID":"8bc0fd80-e941-4202-88f4-0f21f0ca35a4","Type":"ContainerDied","Data":"7b0841c51d3d6891e3482f8e66041e923f12c5bfe4d187e422bf7da8d6c0e35f"}
Jul 28 14:47:51.710946 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.710906    2584 generic.go:356] "Generic (PLEG): container finished" podID="a35f8981-c633-4bea-9e5f-ec0f53daf81d" containerID="c4eb8ac70e99401f1c9fd225fad77bd40ba8546804f989f814ef8c592932aaaa" exitCode=0
Jul 28 14:47:51.711145 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.710991    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg" event={"ID":"a35f8981-c633-4bea-9e5f-ec0f53daf81d","Type":"ContainerDied","Data":"c4eb8ac70e99401f1c9fd225fad77bd40ba8546804f989f814ef8c592932aaaa"}
Jul 28 14:47:51.713085 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.713012    2584 generic.go:356] "Generic (PLEG): container finished" podID="49ce74c8-f4fb-4a06-953e-7f23354d9e24" containerID="6a5b496821558b8cdc1f4f33f1054af5cef3791308c9c4aa00053d38b5f3870f" exitCode=0
Jul 28 14:47:51.713219 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.713085    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq" event={"ID":"49ce74c8-f4fb-4a06-953e-7f23354d9e24","Type":"ContainerDied","Data":"6a5b496821558b8cdc1f4f33f1054af5cef3791308c9c4aa00053d38b5f3870f"}
Jul 28 14:47:51.835966 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.835940    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"
Jul 28 14:47:51.872210 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.872184    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg"
Jul 28 14:47:51.879811 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.879790    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8bl9\" (UniqueName: \"kubernetes.io/projected/8bc0fd80-e941-4202-88f4-0f21f0ca35a4-kube-api-access-m8bl9\") pod \"8bc0fd80-e941-4202-88f4-0f21f0ca35a4\" (UID: \"8bc0fd80-e941-4202-88f4-0f21f0ca35a4\") "
Jul 28 14:47:51.879918 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.879863    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8bc0fd80-e941-4202-88f4-0f21f0ca35a4-maas-api-tls\") pod \"8bc0fd80-e941-4202-88f4-0f21f0ca35a4\" (UID: \"8bc0fd80-e941-4202-88f4-0f21f0ca35a4\") "
Jul 28 14:47:51.879918 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.879905    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bc0fd80-e941-4202-88f4-0f21f0ca35a4-openshift-service-ca\") pod \"8bc0fd80-e941-4202-88f4-0f21f0ca35a4\" (UID: \"8bc0fd80-e941-4202-88f4-0f21f0ca35a4\") "
Jul 28 14:47:51.880332 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.880302    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bc0fd80-e941-4202-88f4-0f21f0ca35a4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8bc0fd80-e941-4202-88f4-0f21f0ca35a4" (UID: "8bc0fd80-e941-4202-88f4-0f21f0ca35a4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:47:51.881887 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.881861    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bc0fd80-e941-4202-88f4-0f21f0ca35a4-kube-api-access-m8bl9" (OuterVolumeSpecName: "kube-api-access-m8bl9") pod "8bc0fd80-e941-4202-88f4-0f21f0ca35a4" (UID: "8bc0fd80-e941-4202-88f4-0f21f0ca35a4"). InnerVolumeSpecName "kube-api-access-m8bl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:47:51.881973 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.881906    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bc0fd80-e941-4202-88f4-0f21f0ca35a4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8bc0fd80-e941-4202-88f4-0f21f0ca35a4" (UID: "8bc0fd80-e941-4202-88f4-0f21f0ca35a4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:47:51.887365 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.887346    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq"
Jul 28 14:47:51.921135 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.921105    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"]
Jul 28 14:47:51.921332 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.921305    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv" podUID="7ea36761-2127-46f2-8362-7420e680d39c" containerName="istio-proxy" containerID="cri-o://8091fdea9d218983a93ee81bafd0f62771e0e4f83eb15e47075f3a47570fe287" gracePeriod=30
Jul 28 14:47:51.980492 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.980460    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49ce74c8-f4fb-4a06-953e-7f23354d9e24-plugins-config-volume\") pod \"49ce74c8-f4fb-4a06-953e-7f23354d9e24\" (UID: \"49ce74c8-f4fb-4a06-953e-7f23354d9e24\") "
Jul 28 14:47:51.980700 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.980514    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hj28f\" (UniqueName: \"kubernetes.io/projected/a35f8981-c633-4bea-9e5f-ec0f53daf81d-kube-api-access-hj28f\") pod \"a35f8981-c633-4bea-9e5f-ec0f53daf81d\" (UID: \"a35f8981-c633-4bea-9e5f-ec0f53daf81d\") "
Jul 28 14:47:51.980700 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.980532    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkrc7\" (UniqueName: \"kubernetes.io/projected/49ce74c8-f4fb-4a06-953e-7f23354d9e24-kube-api-access-gkrc7\") pod \"49ce74c8-f4fb-4a06-953e-7f23354d9e24\" (UID: \"49ce74c8-f4fb-4a06-953e-7f23354d9e24\") "
Jul 28 14:47:51.980700 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.980568    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a35f8981-c633-4bea-9e5f-ec0f53daf81d-plugins-config-volume\") pod \"a35f8981-c633-4bea-9e5f-ec0f53daf81d\" (UID: \"a35f8981-c633-4bea-9e5f-ec0f53daf81d\") "
Jul 28 14:47:51.980868 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.980807    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8bl9\" (UniqueName: \"kubernetes.io/projected/8bc0fd80-e941-4202-88f4-0f21f0ca35a4-kube-api-access-m8bl9\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:51.980868 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.980827    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8bc0fd80-e941-4202-88f4-0f21f0ca35a4-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:51.980868 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.980841    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bc0fd80-e941-4202-88f4-0f21f0ca35a4-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:51.980999 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.980920    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49ce74c8-f4fb-4a06-953e-7f23354d9e24-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "49ce74c8-f4fb-4a06-953e-7f23354d9e24" (UID: "49ce74c8-f4fb-4a06-953e-7f23354d9e24"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:47:51.981038 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.980997    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a35f8981-c633-4bea-9e5f-ec0f53daf81d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a35f8981-c633-4bea-9e5f-ec0f53daf81d" (UID: "a35f8981-c633-4bea-9e5f-ec0f53daf81d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:47:51.982718 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.982692    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49ce74c8-f4fb-4a06-953e-7f23354d9e24-kube-api-access-gkrc7" (OuterVolumeSpecName: "kube-api-access-gkrc7") pod "49ce74c8-f4fb-4a06-953e-7f23354d9e24" (UID: "49ce74c8-f4fb-4a06-953e-7f23354d9e24"). InnerVolumeSpecName "kube-api-access-gkrc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:47:51.982808 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:51.982720    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a35f8981-c633-4bea-9e5f-ec0f53daf81d-kube-api-access-hj28f" (OuterVolumeSpecName: "kube-api-access-hj28f") pod "a35f8981-c633-4bea-9e5f-ec0f53daf81d" (UID: "a35f8981-c633-4bea-9e5f-ec0f53daf81d"). InnerVolumeSpecName "kube-api-access-hj28f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:47:52.081675 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.081575    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a35f8981-c633-4bea-9e5f-ec0f53daf81d-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:52.081675 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.081618    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49ce74c8-f4fb-4a06-953e-7f23354d9e24-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:52.081675 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.081628    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hj28f\" (UniqueName: \"kubernetes.io/projected/a35f8981-c633-4bea-9e5f-ec0f53daf81d-kube-api-access-hj28f\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:52.081675 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.081637    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkrc7\" (UniqueName: \"kubernetes.io/projected/49ce74c8-f4fb-4a06-953e-7f23354d9e24-kube-api-access-gkrc7\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:52.718566 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.718533    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"
Jul 28 14:47:52.719026 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.718534    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287" event={"ID":"8bc0fd80-e941-4202-88f4-0f21f0ca35a4","Type":"ContainerDied","Data":"a4c725afddb33203c4ee79a1da2b3536541934424a0d499b49debe495a70a88e"}
Jul 28 14:47:52.719026 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.718672    2584 scope.go:117] "RemoveContainer" containerID="7b0841c51d3d6891e3482f8e66041e923f12c5bfe4d187e422bf7da8d6c0e35f"
Jul 28 14:47:52.719857 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.719818    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg" event={"ID":"a35f8981-c633-4bea-9e5f-ec0f53daf81d","Type":"ContainerDied","Data":"c441d70a6b866cf392b8b2b8b95e200d29be9842d7841b831c26fb068cd6059c"}
Jul 28 14:47:52.719928 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.719824    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg"
Jul 28 14:47:52.721206 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.721169    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq" event={"ID":"49ce74c8-f4fb-4a06-953e-7f23354d9e24","Type":"ContainerDied","Data":"d82bf8869c644aea0c905a4e1c222371deef1406c407d4d4dadd799a907c27d6"}
Jul 28 14:47:52.721206 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.721186    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq"
Jul 28 14:47:52.729803 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.729778    2584 scope.go:117] "RemoveContainer" containerID="c4eb8ac70e99401f1c9fd225fad77bd40ba8546804f989f814ef8c592932aaaa"
Jul 28 14:47:52.744823 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.744801    2584 scope.go:117] "RemoveContainer" containerID="6a5b496821558b8cdc1f4f33f1054af5cef3791308c9c4aa00053d38b5f3870f"
Jul 28 14:47:52.755772 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.755747    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"]
Jul 28 14:47:52.764535 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.764507    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-35b9d8ff-77f7fdd9dd-hr287"]
Jul 28 14:47:52.774470 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.774393    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq"]
Jul 28 14:47:52.777506 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.777482    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-35b9d8ff-696674d74d-7nxxq"]
Jul 28 14:47:52.796549 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.796520    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49ce74c8-f4fb-4a06-953e-7f23354d9e24" path="/var/lib/kubelet/pods/49ce74c8-f4fb-4a06-953e-7f23354d9e24/volumes"
Jul 28 14:47:52.796902 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.796888    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bc0fd80-e941-4202-88f4-0f21f0ca35a4" path="/var/lib/kubelet/pods/8bc0fd80-e941-4202-88f4-0f21f0ca35a4/volumes"
Jul 28 14:47:52.797167 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.797155    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg"]
Jul 28 14:47:52.797216 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:52.797179    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-35b9d8ff-7dddd8cdd9-mpvfg"]
Jul 28 14:47:54.368257 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:54.368225    2584 patch_prober.go:28] interesting pod/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 28 14:47:54.368622 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:54.368291    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv" podUID="7ea36761-2127-46f2-8362-7420e680d39c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 14:47:54.795975 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:54.795891    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a35f8981-c633-4bea-9e5f-ec0f53daf81d" path="/var/lib/kubelet/pods/a35f8981-c633-4bea-9e5f-ec0f53daf81d/volumes"
Jul 28 14:47:55.584886 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.584853    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"]
Jul 28 14:47:55.585260 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.585226    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bc0fd80-e941-4202-88f4-0f21f0ca35a4" containerName="maas-api"
Jul 28 14:47:55.585260 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.585237    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bc0fd80-e941-4202-88f4-0f21f0ca35a4" containerName="maas-api"
Jul 28 14:47:55.585260 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.585246    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a35f8981-c633-4bea-9e5f-ec0f53daf81d" containerName="payload-processing"
Jul 28 14:47:55.585260 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.585251    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a35f8981-c633-4bea-9e5f-ec0f53daf81d" containerName="payload-processing"
Jul 28 14:47:55.585260 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.585259    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49ce74c8-f4fb-4a06-953e-7f23354d9e24" containerName="payload-pre-processing"
Jul 28 14:47:55.585413 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.585264    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="49ce74c8-f4fb-4a06-953e-7f23354d9e24" containerName="payload-pre-processing"
Jul 28 14:47:55.585413 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.585313    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a35f8981-c633-4bea-9e5f-ec0f53daf81d" containerName="payload-processing"
Jul 28 14:47:55.585413 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.585320    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="49ce74c8-f4fb-4a06-953e-7f23354d9e24" containerName="payload-pre-processing"
Jul 28 14:47:55.585413 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.585329    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bc0fd80-e941-4202-88f4-0f21f0ca35a4" containerName="maas-api"
Jul 28 14:47:55.589861 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.589844    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"
Jul 28 14:47:55.593292 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.593269    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-1bf67f1c-serving-cert\""
Jul 28 14:47:55.601860 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.601833    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"]
Jul 28 14:47:55.714188 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.714154    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab58f40b-b87a-446f-add3-f64375b093cb-maas-api-tls\") pod \"maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj\" (UID: \"ab58f40b-b87a-446f-add3-f64375b093cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"
Jul 28 14:47:55.714359 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.714199    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr6qw\" (UniqueName: \"kubernetes.io/projected/ab58f40b-b87a-446f-add3-f64375b093cb-kube-api-access-jr6qw\") pod \"maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj\" (UID: \"ab58f40b-b87a-446f-add3-f64375b093cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"
Jul 28 14:47:55.714359 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.714270    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab58f40b-b87a-446f-add3-f64375b093cb-openshift-service-ca\") pod \"maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj\" (UID: \"ab58f40b-b87a-446f-add3-f64375b093cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"
Jul 28 14:47:55.815417 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.815385    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab58f40b-b87a-446f-add3-f64375b093cb-maas-api-tls\") pod \"maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj\" (UID: \"ab58f40b-b87a-446f-add3-f64375b093cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"
Jul 28 14:47:55.815631 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.815431    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr6qw\" (UniqueName: \"kubernetes.io/projected/ab58f40b-b87a-446f-add3-f64375b093cb-kube-api-access-jr6qw\") pod \"maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj\" (UID: \"ab58f40b-b87a-446f-add3-f64375b093cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"
Jul 28 14:47:55.815631 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.815497    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab58f40b-b87a-446f-add3-f64375b093cb-openshift-service-ca\") pod \"maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj\" (UID: \"ab58f40b-b87a-446f-add3-f64375b093cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"
Jul 28 14:47:55.815631 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:47:55.815537    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-serving-cert: secret "maas-api-e2e-ait-1bf67f1c-serving-cert" not found
Jul 28 14:47:55.815631 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:47:55.815607    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab58f40b-b87a-446f-add3-f64375b093cb-maas-api-tls podName:ab58f40b-b87a-446f-add3-f64375b093cb nodeName:}" failed. No retries permitted until 2026-07-28 14:47:56.315586078 +0000 UTC m=+1746.117128286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ab58f40b-b87a-446f-add3-f64375b093cb-maas-api-tls") pod "maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj" (UID: "ab58f40b-b87a-446f-add3-f64375b093cb") : secret "maas-api-e2e-ait-1bf67f1c-serving-cert" not found
Jul 28 14:47:55.816123 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.816082    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab58f40b-b87a-446f-add3-f64375b093cb-openshift-service-ca\") pod \"maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj\" (UID: \"ab58f40b-b87a-446f-add3-f64375b093cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"
Jul 28 14:47:55.824074 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:55.824042    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr6qw\" (UniqueName: \"kubernetes.io/projected/ab58f40b-b87a-446f-add3-f64375b093cb-kube-api-access-jr6qw\") pod \"maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj\" (UID: \"ab58f40b-b87a-446f-add3-f64375b093cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"
Jul 28 14:47:56.319077 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.319043    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab58f40b-b87a-446f-add3-f64375b093cb-maas-api-tls\") pod \"maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj\" (UID: \"ab58f40b-b87a-446f-add3-f64375b093cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"
Jul 28 14:47:56.321627 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.321605    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab58f40b-b87a-446f-add3-f64375b093cb-maas-api-tls\") pod \"maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj\" (UID: \"ab58f40b-b87a-446f-add3-f64375b093cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"
Jul 28 14:47:56.397381 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.397347    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj"]
Jul 28 14:47:56.400910 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.400893    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj"
Jul 28 14:47:56.403253 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.403237    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-1bf67f1c\""
Jul 28 14:47:56.412710 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.412688    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj"]
Jul 28 14:47:56.432274 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.432240    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959"]
Jul 28 14:47:56.436069 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.436050    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959"
Jul 28 14:47:56.438652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.438632    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-1bf67f1c-dockercfg-48btg\""
Jul 28 14:47:56.447821 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.447800    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959"]
Jul 28 14:47:56.501590 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.501559    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"
Jul 28 14:47:56.522728 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.521633    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffd0f25f-d271-4f53-9783-b185d14c708e-plugins-config-volume\") pod \"payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959\" (UID: \"ffd0f25f-d271-4f53-9783-b185d14c708e\") " pod="openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959"
Jul 28 14:47:56.522728 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.521765    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7827\" (UniqueName: \"kubernetes.io/projected/ffd0f25f-d271-4f53-9783-b185d14c708e-kube-api-access-f7827\") pod \"payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959\" (UID: \"ffd0f25f-d271-4f53-9783-b185d14c708e\") " pod="openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959"
Jul 28 14:47:56.522728 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.521991    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4d87\" (UniqueName: \"kubernetes.io/projected/cfb4015c-db07-462d-83ba-b3358a73351d-kube-api-access-b4d87\") pod \"payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj\" (UID: \"cfb4015c-db07-462d-83ba-b3358a73351d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj"
Jul 28 14:47:56.522728 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.522081    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfb4015c-db07-462d-83ba-b3358a73351d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj\" (UID: \"cfb4015c-db07-462d-83ba-b3358a73351d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj"
Jul 28 14:47:56.623791 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.623760    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffd0f25f-d271-4f53-9783-b185d14c708e-plugins-config-volume\") pod \"payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959\" (UID: \"ffd0f25f-d271-4f53-9783-b185d14c708e\") " pod="openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959"
Jul 28 14:47:56.623791 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.623791    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7827\" (UniqueName: \"kubernetes.io/projected/ffd0f25f-d271-4f53-9783-b185d14c708e-kube-api-access-f7827\") pod \"payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959\" (UID: \"ffd0f25f-d271-4f53-9783-b185d14c708e\") " pod="openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959"
Jul 28 14:47:56.624313 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.623847    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4d87\" (UniqueName: \"kubernetes.io/projected/cfb4015c-db07-462d-83ba-b3358a73351d-kube-api-access-b4d87\") pod \"payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj\" (UID: \"cfb4015c-db07-462d-83ba-b3358a73351d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj"
Jul 28 14:47:56.624313 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.623891    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfb4015c-db07-462d-83ba-b3358a73351d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj\" (UID: \"cfb4015c-db07-462d-83ba-b3358a73351d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj"
Jul 28 14:47:56.624624 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.624604    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfb4015c-db07-462d-83ba-b3358a73351d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj\" (UID: \"cfb4015c-db07-462d-83ba-b3358a73351d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj"
Jul 28 14:47:56.624818 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.624786    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffd0f25f-d271-4f53-9783-b185d14c708e-plugins-config-volume\") pod \"payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959\" (UID: \"ffd0f25f-d271-4f53-9783-b185d14c708e\") " pod="openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959"
Jul 28 14:47:56.633014 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.632989    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4d87\" (UniqueName: \"kubernetes.io/projected/cfb4015c-db07-462d-83ba-b3358a73351d-kube-api-access-b4d87\") pod \"payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj\" (UID: \"cfb4015c-db07-462d-83ba-b3358a73351d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj"
Jul 28 14:47:56.633014 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.633007    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7827\" (UniqueName: \"kubernetes.io/projected/ffd0f25f-d271-4f53-9783-b185d14c708e-kube-api-access-f7827\") pod \"payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959\" (UID: \"ffd0f25f-d271-4f53-9783-b185d14c708e\") " pod="openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959"
Jul 28 14:47:56.653922 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.653840    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"]
Jul 28 14:47:56.712016 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.711980    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj"
Jul 28 14:47:56.744853 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.744820    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj" event={"ID":"ab58f40b-b87a-446f-add3-f64375b093cb","Type":"ContainerStarted","Data":"51dc241378be967f7f7c7d673785d25e883cc49c556635f8454fc2ccb2674cdc"}
Jul 28 14:47:56.745991 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.745965    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959"
Jul 28 14:47:56.867973 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.867859    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj"]
Jul 28 14:47:56.869417 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:47:56.869340    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfb4015c_db07_462d_83ba_b3358a73351d.slice/crio-f97ba9c7e7ea15448d0dc3f28cd6662255a3ffd38af74343a450e50ea758099f WatchSource:0}: Error finding container f97ba9c7e7ea15448d0dc3f28cd6662255a3ffd38af74343a450e50ea758099f: Status 404 returned error can't find the container with id f97ba9c7e7ea15448d0dc3f28cd6662255a3ffd38af74343a450e50ea758099f
Jul 28 14:47:56.890948 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:56.890925    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959"]
Jul 28 14:47:56.893243 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:47:56.893218    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffd0f25f_d271_4f53_9783_b185d14c708e.slice/crio-24152a3d4c88e3079e430fcfb3a91cfab5c0f9cdba9ea8c5ac421543cdbb930e WatchSource:0}: Error finding container 24152a3d4c88e3079e430fcfb3a91cfab5c0f9cdba9ea8c5ac421543cdbb930e: Status 404 returned error can't find the container with id 24152a3d4c88e3079e430fcfb3a91cfab5c0f9cdba9ea8c5ac421543cdbb930e
Jul 28 14:47:57.167455 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.167435    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:57.230689 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.230656    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ea36761-2127-46f2-8362-7420e680d39c-istio-podinfo\") pod \"7ea36761-2127-46f2-8362-7420e680d39c\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") "
Jul 28 14:47:57.230870 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.230707    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-istio-data\") pod \"7ea36761-2127-46f2-8362-7420e680d39c\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") "
Jul 28 14:47:57.230870 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.230731    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ea36761-2127-46f2-8362-7420e680d39c-istio-token\") pod \"7ea36761-2127-46f2-8362-7420e680d39c\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") "
Jul 28 14:47:57.230870 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.230763    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-istio-envoy\") pod \"7ea36761-2127-46f2-8362-7420e680d39c\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") "
Jul 28 14:47:57.230870 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.230810    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-workload-certs\") pod \"7ea36761-2127-46f2-8362-7420e680d39c\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") "
Jul 28 14:47:57.230870 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.230838    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-credential-socket\") pod \"7ea36761-2127-46f2-8362-7420e680d39c\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") "
Jul 28 14:47:57.230870 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.230865    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zjtd\" (UniqueName: \"kubernetes.io/projected/7ea36761-2127-46f2-8362-7420e680d39c-kube-api-access-7zjtd\") pod \"7ea36761-2127-46f2-8362-7420e680d39c\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") "
Jul 28 14:47:57.231203 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.230931    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-workload-socket\") pod \"7ea36761-2127-46f2-8362-7420e680d39c\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") "
Jul 28 14:47:57.231203 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.230995    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ea36761-2127-46f2-8362-7420e680d39c-istiod-ca-cert\") pod \"7ea36761-2127-46f2-8362-7420e680d39c\" (UID: \"7ea36761-2127-46f2-8362-7420e680d39c\") "
Jul 28 14:47:57.232013 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.231487    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7ea36761-2127-46f2-8362-7420e680d39c" (UID: "7ea36761-2127-46f2-8362-7420e680d39c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:47:57.232013 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.231774    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-istio-data" (OuterVolumeSpecName: "istio-data") pod "7ea36761-2127-46f2-8362-7420e680d39c" (UID: "7ea36761-2127-46f2-8362-7420e680d39c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:47:57.232220 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.232056    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7ea36761-2127-46f2-8362-7420e680d39c" (UID: "7ea36761-2127-46f2-8362-7420e680d39c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:47:57.232220 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.231980    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7ea36761-2127-46f2-8362-7420e680d39c" (UID: "7ea36761-2127-46f2-8362-7420e680d39c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:47:57.232509 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.232481    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ea36761-2127-46f2-8362-7420e680d39c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7ea36761-2127-46f2-8362-7420e680d39c" (UID: "7ea36761-2127-46f2-8362-7420e680d39c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:47:57.233384 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.233341    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7ea36761-2127-46f2-8362-7420e680d39c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7ea36761-2127-46f2-8362-7420e680d39c" (UID: "7ea36761-2127-46f2-8362-7420e680d39c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 14:47:57.233838 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.233809    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ea36761-2127-46f2-8362-7420e680d39c-kube-api-access-7zjtd" (OuterVolumeSpecName: "kube-api-access-7zjtd") pod "7ea36761-2127-46f2-8362-7420e680d39c" (UID: "7ea36761-2127-46f2-8362-7420e680d39c"). InnerVolumeSpecName "kube-api-access-7zjtd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:47:57.234085 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.234061    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7ea36761-2127-46f2-8362-7420e680d39c" (UID: "7ea36761-2127-46f2-8362-7420e680d39c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:47:57.234429 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.234400    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ea36761-2127-46f2-8362-7420e680d39c-istio-token" (OuterVolumeSpecName: "istio-token") pod "7ea36761-2127-46f2-8362-7420e680d39c" (UID: "7ea36761-2127-46f2-8362-7420e680d39c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:47:57.332423 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.332383    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ea36761-2127-46f2-8362-7420e680d39c-istio-podinfo\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:57.332423 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.332422    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-istio-data\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:57.332681 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.332435    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ea36761-2127-46f2-8362-7420e680d39c-istio-token\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:57.332681 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.332447    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-istio-envoy\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:57.332681 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.332460    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-workload-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:57.332681 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.332472    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-credential-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:57.332681 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.332486    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zjtd\" (UniqueName: \"kubernetes.io/projected/7ea36761-2127-46f2-8362-7420e680d39c-kube-api-access-7zjtd\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:57.332681 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.332500    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ea36761-2127-46f2-8362-7420e680d39c-workload-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:57.332681 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.332514    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ea36761-2127-46f2-8362-7420e680d39c-istiod-ca-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:47:57.755170 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.755134    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj" event={"ID":"ab58f40b-b87a-446f-add3-f64375b093cb","Type":"ContainerStarted","Data":"9a347b419ebb589e7501296a554c10a800e80ed1c04e4e4c47e1b86303600036"}
Jul 28 14:47:57.755691 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.755272    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"
Jul 28 14:47:57.756862 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.756837    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj" event={"ID":"cfb4015c-db07-462d-83ba-b3358a73351d","Type":"ContainerStarted","Data":"ff24bbf138150705ec599304933554667a35376be81da463c7636779160406f9"}
Jul 28 14:47:57.756989 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.756883    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj" event={"ID":"cfb4015c-db07-462d-83ba-b3358a73351d","Type":"ContainerStarted","Data":"f97ba9c7e7ea15448d0dc3f28cd6662255a3ffd38af74343a450e50ea758099f"}
Jul 28 14:47:57.756989 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.756923    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj"
Jul 28 14:47:57.758541 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.758516    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959" event={"ID":"ffd0f25f-d271-4f53-9783-b185d14c708e","Type":"ContainerStarted","Data":"07a93acf5ff11537aca43760577827f039eded1ff3da84cee0a722b4a6de9864"}
Jul 28 14:47:57.758677 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.758563    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959" event={"ID":"ffd0f25f-d271-4f53-9783-b185d14c708e","Type":"ContainerStarted","Data":"24152a3d4c88e3079e430fcfb3a91cfab5c0f9cdba9ea8c5ac421543cdbb930e"}
Jul 28 14:47:57.758677 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.758593    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959"
Jul 28 14:47:57.760641 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.760620    2584 generic.go:356] "Generic (PLEG): container finished" podID="7ea36761-2127-46f2-8362-7420e680d39c" containerID="8091fdea9d218983a93ee81bafd0f62771e0e4f83eb15e47075f3a47570fe287" exitCode=0
Jul 28 14:47:57.760728 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.760651    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv" event={"ID":"7ea36761-2127-46f2-8362-7420e680d39c","Type":"ContainerDied","Data":"8091fdea9d218983a93ee81bafd0f62771e0e4f83eb15e47075f3a47570fe287"}
Jul 28 14:47:57.760728 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.760682    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"
Jul 28 14:47:57.760728 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.760697    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv" event={"ID":"7ea36761-2127-46f2-8362-7420e680d39c","Type":"ContainerDied","Data":"5dc7dec006e5f4691168f337ca66fa0624374b4b9f9c684ce515220f32d89995"}
Jul 28 14:47:57.760728 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.760725    2584 scope.go:117] "RemoveContainer" containerID="8091fdea9d218983a93ee81bafd0f62771e0e4f83eb15e47075f3a47570fe287"
Jul 28 14:47:57.769971 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.769951    2584 scope.go:117] "RemoveContainer" containerID="77447163aba9a82ab04719f331d94aee9a9198476f48e858d94c95fba748696a"
Jul 28 14:47:57.772953 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.772908    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj" podStartSLOduration=2.331945631 podStartE2EDuration="2.772892723s" totalImagesPullingTime="440.947092ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:47:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:47:57.771147897 +0000 UTC m=+1747.572690110" watchObservedRunningTime="2026-07-28 14:47:57.772892723 +0000 UTC m=+1747.574434936"
Jul 28 14:47:57.779977 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.779960    2584 scope.go:117] "RemoveContainer" containerID="8091fdea9d218983a93ee81bafd0f62771e0e4f83eb15e47075f3a47570fe287"
Jul 28 14:47:57.780292 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:47:57.780271    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8091fdea9d218983a93ee81bafd0f62771e0e4f83eb15e47075f3a47570fe287\": container with ID starting with 8091fdea9d218983a93ee81bafd0f62771e0e4f83eb15e47075f3a47570fe287 not found: ID does not exist" containerID="8091fdea9d218983a93ee81bafd0f62771e0e4f83eb15e47075f3a47570fe287"
Jul 28 14:47:57.780346 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.780303    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8091fdea9d218983a93ee81bafd0f62771e0e4f83eb15e47075f3a47570fe287"} err="failed to get container status \"8091fdea9d218983a93ee81bafd0f62771e0e4f83eb15e47075f3a47570fe287\": rpc error: code = NotFound desc = could not find container \"8091fdea9d218983a93ee81bafd0f62771e0e4f83eb15e47075f3a47570fe287\": container with ID starting with 8091fdea9d218983a93ee81bafd0f62771e0e4f83eb15e47075f3a47570fe287 not found: ID does not exist"
Jul 28 14:47:57.780346 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.780320    2584 scope.go:117] "RemoveContainer" containerID="77447163aba9a82ab04719f331d94aee9a9198476f48e858d94c95fba748696a"
Jul 28 14:47:57.780574 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:47:57.780555    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77447163aba9a82ab04719f331d94aee9a9198476f48e858d94c95fba748696a\": container with ID starting with 77447163aba9a82ab04719f331d94aee9a9198476f48e858d94c95fba748696a not found: ID does not exist" containerID="77447163aba9a82ab04719f331d94aee9a9198476f48e858d94c95fba748696a"
Jul 28 14:47:57.780613 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.780581    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77447163aba9a82ab04719f331d94aee9a9198476f48e858d94c95fba748696a"} err="failed to get container status \"77447163aba9a82ab04719f331d94aee9a9198476f48e858d94c95fba748696a\": rpc error: code = NotFound desc = could not find container \"77447163aba9a82ab04719f331d94aee9a9198476f48e858d94c95fba748696a\": container with ID starting with 77447163aba9a82ab04719f331d94aee9a9198476f48e858d94c95fba748696a not found: ID does not exist"
Jul 28 14:47:57.785879 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.785855    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"]
Jul 28 14:47:57.791919 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.791894    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-35b9d8ff-gw-openshift-default-677dcbbb8d-57pmv"]
Jul 28 14:47:57.808066 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.808019    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj" podStartSLOduration=1.8080054250000002 podStartE2EDuration="1.808005425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:47:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:47:57.806430143 +0000 UTC m=+1747.607972359" watchObservedRunningTime="2026-07-28 14:47:57.808005425 +0000 UTC m=+1747.609547638"
Jul 28 14:47:57.824437 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:57.824251    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959" podStartSLOduration=1.8242339319999998 podStartE2EDuration="1.824233932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:47:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:47:57.823939987 +0000 UTC m=+1747.625482213" watchObservedRunningTime="2026-07-28 14:47:57.824233932 +0000 UTC m=+1747.625776147"
Jul 28 14:47:58.796946 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:47:58.796917    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ea36761-2127-46f2-8362-7420e680d39c" path="/var/lib/kubelet/pods/7ea36761-2127-46f2-8362-7420e680d39c/volumes"
Jul 28 14:48:03.775014 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:03.774978    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"
Jul 28 14:48:08.771141 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:08.771086    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959"
Jul 28 14:48:08.771570 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:08.771188    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj"
Jul 28 14:48:26.195878 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.195836    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"]
Jul 28 14:48:26.196421 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.196074    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj" podUID="ab58f40b-b87a-446f-add3-f64375b093cb" containerName="maas-api" containerID="cri-o://9a347b419ebb589e7501296a554c10a800e80ed1c04e4e4c47e1b86303600036" gracePeriod=30
Jul 28 14:48:26.250905 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.250870    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959"]
Jul 28 14:48:26.251164 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.251121    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959" podUID="ffd0f25f-d271-4f53-9783-b185d14c708e" containerName="payload-processing" containerID="cri-o://07a93acf5ff11537aca43760577827f039eded1ff3da84cee0a722b4a6de9864" gracePeriod=30
Jul 28 14:48:26.256008 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.255981    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj"]
Jul 28 14:48:26.256281 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.256240    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj" podUID="cfb4015c-db07-462d-83ba-b3358a73351d" containerName="payload-pre-processing" containerID="cri-o://ff24bbf138150705ec599304933554667a35376be81da463c7636779160406f9" gracePeriod=30
Jul 28 14:48:26.524955 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.524929    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"
Jul 28 14:48:26.540915 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.540890    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj"
Jul 28 14:48:26.545281 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.545254    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959"
Jul 28 14:48:26.589496 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.589470    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab58f40b-b87a-446f-add3-f64375b093cb-openshift-service-ca\") pod \"ab58f40b-b87a-446f-add3-f64375b093cb\" (UID: \"ab58f40b-b87a-446f-add3-f64375b093cb\") "
Jul 28 14:48:26.589683 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.589555    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7827\" (UniqueName: \"kubernetes.io/projected/ffd0f25f-d271-4f53-9783-b185d14c708e-kube-api-access-f7827\") pod \"ffd0f25f-d271-4f53-9783-b185d14c708e\" (UID: \"ffd0f25f-d271-4f53-9783-b185d14c708e\") "
Jul 28 14:48:26.589683 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.589616    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffd0f25f-d271-4f53-9783-b185d14c708e-plugins-config-volume\") pod \"ffd0f25f-d271-4f53-9783-b185d14c708e\" (UID: \"ffd0f25f-d271-4f53-9783-b185d14c708e\") "
Jul 28 14:48:26.589683 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.589635    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4d87\" (UniqueName: \"kubernetes.io/projected/cfb4015c-db07-462d-83ba-b3358a73351d-kube-api-access-b4d87\") pod \"cfb4015c-db07-462d-83ba-b3358a73351d\" (UID: \"cfb4015c-db07-462d-83ba-b3358a73351d\") "
Jul 28 14:48:26.589683 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.589656    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfb4015c-db07-462d-83ba-b3358a73351d-plugins-config-volume\") pod \"cfb4015c-db07-462d-83ba-b3358a73351d\" (UID: \"cfb4015c-db07-462d-83ba-b3358a73351d\") "
Jul 28 14:48:26.589917 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.589770    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab58f40b-b87a-446f-add3-f64375b093cb-maas-api-tls\") pod \"ab58f40b-b87a-446f-add3-f64375b093cb\" (UID: \"ab58f40b-b87a-446f-add3-f64375b093cb\") "
Jul 28 14:48:26.589917 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.589830    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jr6qw\" (UniqueName: \"kubernetes.io/projected/ab58f40b-b87a-446f-add3-f64375b093cb-kube-api-access-jr6qw\") pod \"ab58f40b-b87a-446f-add3-f64375b093cb\" (UID: \"ab58f40b-b87a-446f-add3-f64375b093cb\") "
Jul 28 14:48:26.590022 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.589910    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab58f40b-b87a-446f-add3-f64375b093cb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ab58f40b-b87a-446f-add3-f64375b093cb" (UID: "ab58f40b-b87a-446f-add3-f64375b093cb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:48:26.590081 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.590020    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffd0f25f-d271-4f53-9783-b185d14c708e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ffd0f25f-d271-4f53-9783-b185d14c708e" (UID: "ffd0f25f-d271-4f53-9783-b185d14c708e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:48:26.590157 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.590023    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfb4015c-db07-462d-83ba-b3358a73351d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cfb4015c-db07-462d-83ba-b3358a73351d" (UID: "cfb4015c-db07-462d-83ba-b3358a73351d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:48:26.590214 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.590184    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab58f40b-b87a-446f-add3-f64375b093cb-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:48:26.590214 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.590199    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffd0f25f-d271-4f53-9783-b185d14c708e-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:48:26.590214 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.590208    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfb4015c-db07-462d-83ba-b3358a73351d-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:48:26.591967 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.591942    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffd0f25f-d271-4f53-9783-b185d14c708e-kube-api-access-f7827" (OuterVolumeSpecName: "kube-api-access-f7827") pod "ffd0f25f-d271-4f53-9783-b185d14c708e" (UID: "ffd0f25f-d271-4f53-9783-b185d14c708e"). InnerVolumeSpecName "kube-api-access-f7827". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:48:26.591967 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.591941    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab58f40b-b87a-446f-add3-f64375b093cb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ab58f40b-b87a-446f-add3-f64375b093cb" (UID: "ab58f40b-b87a-446f-add3-f64375b093cb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:48:26.592340 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.592284    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab58f40b-b87a-446f-add3-f64375b093cb-kube-api-access-jr6qw" (OuterVolumeSpecName: "kube-api-access-jr6qw") pod "ab58f40b-b87a-446f-add3-f64375b093cb" (UID: "ab58f40b-b87a-446f-add3-f64375b093cb"). InnerVolumeSpecName "kube-api-access-jr6qw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:48:26.592418 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.592343    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfb4015c-db07-462d-83ba-b3358a73351d-kube-api-access-b4d87" (OuterVolumeSpecName: "kube-api-access-b4d87") pod "cfb4015c-db07-462d-83ba-b3358a73351d" (UID: "cfb4015c-db07-462d-83ba-b3358a73351d"). InnerVolumeSpecName "kube-api-access-b4d87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:48:26.690697 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.690663    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4d87\" (UniqueName: \"kubernetes.io/projected/cfb4015c-db07-462d-83ba-b3358a73351d-kube-api-access-b4d87\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:48:26.690697 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.690692    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab58f40b-b87a-446f-add3-f64375b093cb-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:48:26.690697 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.690702    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jr6qw\" (UniqueName: \"kubernetes.io/projected/ab58f40b-b87a-446f-add3-f64375b093cb-kube-api-access-jr6qw\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:48:26.690939 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.690711    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7827\" (UniqueName: \"kubernetes.io/projected/ffd0f25f-d271-4f53-9783-b185d14c708e-kube-api-access-f7827\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:48:26.911519 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.911485    2584 generic.go:356] "Generic (PLEG): container finished" podID="ab58f40b-b87a-446f-add3-f64375b093cb" containerID="9a347b419ebb589e7501296a554c10a800e80ed1c04e4e4c47e1b86303600036" exitCode=0
Jul 28 14:48:26.911682 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.911527    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"
Jul 28 14:48:26.911682 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.911570    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj" event={"ID":"ab58f40b-b87a-446f-add3-f64375b093cb","Type":"ContainerDied","Data":"9a347b419ebb589e7501296a554c10a800e80ed1c04e4e4c47e1b86303600036"}
Jul 28 14:48:26.911682 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.911624    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj" event={"ID":"ab58f40b-b87a-446f-add3-f64375b093cb","Type":"ContainerDied","Data":"51dc241378be967f7f7c7d673785d25e883cc49c556635f8454fc2ccb2674cdc"}
Jul 28 14:48:26.911682 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.911652    2584 scope.go:117] "RemoveContainer" containerID="9a347b419ebb589e7501296a554c10a800e80ed1c04e4e4c47e1b86303600036"
Jul 28 14:48:26.912927 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.912894    2584 generic.go:356] "Generic (PLEG): container finished" podID="cfb4015c-db07-462d-83ba-b3358a73351d" containerID="ff24bbf138150705ec599304933554667a35376be81da463c7636779160406f9" exitCode=0
Jul 28 14:48:26.913039 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.912979    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj"
Jul 28 14:48:26.913039 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.912982    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj" event={"ID":"cfb4015c-db07-462d-83ba-b3358a73351d","Type":"ContainerDied","Data":"ff24bbf138150705ec599304933554667a35376be81da463c7636779160406f9"}
Jul 28 14:48:26.913039 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.913031    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj" event={"ID":"cfb4015c-db07-462d-83ba-b3358a73351d","Type":"ContainerDied","Data":"f97ba9c7e7ea15448d0dc3f28cd6662255a3ffd38af74343a450e50ea758099f"}
Jul 28 14:48:26.914610 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.914587    2584 generic.go:356] "Generic (PLEG): container finished" podID="ffd0f25f-d271-4f53-9783-b185d14c708e" containerID="07a93acf5ff11537aca43760577827f039eded1ff3da84cee0a722b4a6de9864" exitCode=0
Jul 28 14:48:26.914726 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.914613    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959" event={"ID":"ffd0f25f-d271-4f53-9783-b185d14c708e","Type":"ContainerDied","Data":"07a93acf5ff11537aca43760577827f039eded1ff3da84cee0a722b4a6de9864"}
Jul 28 14:48:26.914726 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.914638    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959"
Jul 28 14:48:26.914726 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.914661    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959" event={"ID":"ffd0f25f-d271-4f53-9783-b185d14c708e","Type":"ContainerDied","Data":"24152a3d4c88e3079e430fcfb3a91cfab5c0f9cdba9ea8c5ac421543cdbb930e"}
Jul 28 14:48:26.921284 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.921261    2584 scope.go:117] "RemoveContainer" containerID="9a347b419ebb589e7501296a554c10a800e80ed1c04e4e4c47e1b86303600036"
Jul 28 14:48:26.921519 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:48:26.921500    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a347b419ebb589e7501296a554c10a800e80ed1c04e4e4c47e1b86303600036\": container with ID starting with 9a347b419ebb589e7501296a554c10a800e80ed1c04e4e4c47e1b86303600036 not found: ID does not exist" containerID="9a347b419ebb589e7501296a554c10a800e80ed1c04e4e4c47e1b86303600036"
Jul 28 14:48:26.921572 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.921527    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a347b419ebb589e7501296a554c10a800e80ed1c04e4e4c47e1b86303600036"} err="failed to get container status \"9a347b419ebb589e7501296a554c10a800e80ed1c04e4e4c47e1b86303600036\": rpc error: code = NotFound desc = could not find container \"9a347b419ebb589e7501296a554c10a800e80ed1c04e4e4c47e1b86303600036\": container with ID starting with 9a347b419ebb589e7501296a554c10a800e80ed1c04e4e4c47e1b86303600036 not found: ID does not exist"
Jul 28 14:48:26.921572 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.921544    2584 scope.go:117] "RemoveContainer" containerID="ff24bbf138150705ec599304933554667a35376be81da463c7636779160406f9"
Jul 28 14:48:26.928605 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.928592    2584 scope.go:117] "RemoveContainer" containerID="ff24bbf138150705ec599304933554667a35376be81da463c7636779160406f9"
Jul 28 14:48:26.928835 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:48:26.928817    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff24bbf138150705ec599304933554667a35376be81da463c7636779160406f9\": container with ID starting with ff24bbf138150705ec599304933554667a35376be81da463c7636779160406f9 not found: ID does not exist" containerID="ff24bbf138150705ec599304933554667a35376be81da463c7636779160406f9"
Jul 28 14:48:26.928926 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.928844    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff24bbf138150705ec599304933554667a35376be81da463c7636779160406f9"} err="failed to get container status \"ff24bbf138150705ec599304933554667a35376be81da463c7636779160406f9\": rpc error: code = NotFound desc = could not find container \"ff24bbf138150705ec599304933554667a35376be81da463c7636779160406f9\": container with ID starting with ff24bbf138150705ec599304933554667a35376be81da463c7636779160406f9 not found: ID does not exist"
Jul 28 14:48:26.928926 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.928865    2584 scope.go:117] "RemoveContainer" containerID="07a93acf5ff11537aca43760577827f039eded1ff3da84cee0a722b4a6de9864"
Jul 28 14:48:26.933946 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.933920    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj"]
Jul 28 14:48:26.936734 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.936717    2584 scope.go:117] "RemoveContainer" containerID="07a93acf5ff11537aca43760577827f039eded1ff3da84cee0a722b4a6de9864"
Jul 28 14:48:26.937003 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:48:26.936986    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07a93acf5ff11537aca43760577827f039eded1ff3da84cee0a722b4a6de9864\": container with ID starting with 07a93acf5ff11537aca43760577827f039eded1ff3da84cee0a722b4a6de9864 not found: ID does not exist" containerID="07a93acf5ff11537aca43760577827f039eded1ff3da84cee0a722b4a6de9864"
Jul 28 14:48:26.937051 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.937007    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07a93acf5ff11537aca43760577827f039eded1ff3da84cee0a722b4a6de9864"} err="failed to get container status \"07a93acf5ff11537aca43760577827f039eded1ff3da84cee0a722b4a6de9864\": rpc error: code = NotFound desc = could not find container \"07a93acf5ff11537aca43760577827f039eded1ff3da84cee0a722b4a6de9864\": container with ID starting with 07a93acf5ff11537aca43760577827f039eded1ff3da84cee0a722b4a6de9864 not found: ID does not exist"
Jul 28 14:48:26.937560 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.937542    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1bf67f1c-f9cd5f564-pd6bj"]
Jul 28 14:48:26.949769 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.949739    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959"]
Jul 28 14:48:26.951678 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.951660    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1bf67f1c-5fb5d56dd4-97959"]
Jul 28 14:48:26.961810 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.961790    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"]
Jul 28 14:48:26.965724 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:26.965703    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1bf67f1c-6d6d6698cb-l2lkj"]
Jul 28 14:48:28.511753 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.511720    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"]
Jul 28 14:48:28.512137 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.512065    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea36761-2127-46f2-8362-7420e680d39c" containerName="istio-proxy"
Jul 28 14:48:28.512137 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.512076    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea36761-2127-46f2-8362-7420e680d39c" containerName="istio-proxy"
Jul 28 14:48:28.512137 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.512084    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfb4015c-db07-462d-83ba-b3358a73351d" containerName="payload-pre-processing"
Jul 28 14:48:28.512137 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.512106    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfb4015c-db07-462d-83ba-b3358a73351d" containerName="payload-pre-processing"
Jul 28 14:48:28.512137 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.512121    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab58f40b-b87a-446f-add3-f64375b093cb" containerName="maas-api"
Jul 28 14:48:28.512137 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.512126    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab58f40b-b87a-446f-add3-f64375b093cb" containerName="maas-api"
Jul 28 14:48:28.512137 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.512138    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffd0f25f-d271-4f53-9783-b185d14c708e" containerName="payload-processing"
Jul 28 14:48:28.512442 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.512145    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffd0f25f-d271-4f53-9783-b185d14c708e" containerName="payload-processing"
Jul 28 14:48:28.512442 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.512210    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ea36761-2127-46f2-8362-7420e680d39c" containerName="istio-proxy"
Jul 28 14:48:28.512442 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.512220    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab58f40b-b87a-446f-add3-f64375b093cb" containerName="maas-api"
Jul 28 14:48:28.512442 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.512229    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfb4015c-db07-462d-83ba-b3358a73351d" containerName="payload-pre-processing"
Jul 28 14:48:28.512442 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.512239    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffd0f25f-d271-4f53-9783-b185d14c708e" containerName="payload-processing"
Jul 28 14:48:28.516919 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.516898    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.519316 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.519296    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-e9a43c4a-openshift-default-dockercfg-nbkqd\""
Jul 28 14:48:28.529578 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.529556    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"]
Jul 28 14:48:28.605031 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.604995    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wntv6\" (UniqueName: \"kubernetes.io/projected/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-kube-api-access-wntv6\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.605031 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.605035    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-token\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.605257 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.605138    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-envoy\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.605257 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.605178    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-data\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.605257 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.605236    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istiod-ca-cert\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.605365 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.605293    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-podinfo\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.605365 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.605313    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-workload-certs\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.605365 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.605341    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-credential-socket\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.605516 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.605420    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-workload-socket\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.706460 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.706417    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-podinfo\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.706460 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.706464    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-workload-certs\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.706695 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.706501    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-credential-socket\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.706695 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.706521    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-workload-socket\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.706695 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.706545    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wntv6\" (UniqueName: \"kubernetes.io/projected/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-kube-api-access-wntv6\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.706695 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.706571    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-token\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.706695 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.706605    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-envoy\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.706945 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.706716    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-data\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.706945 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.706763    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istiod-ca-cert\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.707029 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.706972    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-workload-socket\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.707029 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.707007    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-credential-socket\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.707124 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.707078    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-data\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.707206 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.707186    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-workload-certs\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.707506 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.707485    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istiod-ca-cert\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.709003 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.708983    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-envoy\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.709164 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.709147    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-podinfo\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.714787 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.714756    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-token\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.714871 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.714817    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wntv6\" (UniqueName: \"kubernetes.io/projected/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-kube-api-access-wntv6\") pod \"e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") " pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.799624 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.799546    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab58f40b-b87a-446f-add3-f64375b093cb" path="/var/lib/kubelet/pods/ab58f40b-b87a-446f-add3-f64375b093cb/volumes"
Jul 28 14:48:28.800062 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.800039    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfb4015c-db07-462d-83ba-b3358a73351d" path="/var/lib/kubelet/pods/cfb4015c-db07-462d-83ba-b3358a73351d/volumes"
Jul 28 14:48:28.800522 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.800505    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffd0f25f-d271-4f53-9783-b185d14c708e" path="/var/lib/kubelet/pods/ffd0f25f-d271-4f53-9783-b185d14c708e/volumes"
Jul 28 14:48:28.827482 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.827447    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:28.988154 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.988126    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"]
Jul 28 14:48:28.990384 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:48:28.990352    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcdb5eaa_8308_494c_a3d6_589e9cb4e2a6.slice/crio-daa7f6e00edb10f857cc1a43548964a0f08d42fa7bfe7edd617fddd7ace32699 WatchSource:0}: Error finding container daa7f6e00edb10f857cc1a43548964a0f08d42fa7bfe7edd617fddd7ace32699: Status 404 returned error can't find the container with id daa7f6e00edb10f857cc1a43548964a0f08d42fa7bfe7edd617fddd7ace32699
Jul 28 14:48:28.992544 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.992511    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:48:28.992610 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.992580    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:48:28.992652 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:28.992607    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:48:29.934424 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:29.934387    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9" event={"ID":"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6","Type":"ContainerStarted","Data":"4329e1bd542d7801470afc73d2bebaacc2ceb15205b4d50f69f0300285d54ac7"}
Jul 28 14:48:29.934797 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:29.934471    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9" event={"ID":"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6","Type":"ContainerStarted","Data":"daa7f6e00edb10f857cc1a43548964a0f08d42fa7bfe7edd617fddd7ace32699"}
Jul 28 14:48:29.957145 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:29.957051    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9" podStartSLOduration=1.957036455 podStartE2EDuration="1.957036455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:48:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:48:29.955456527 +0000 UTC m=+1779.756998753" watchObservedRunningTime="2026-07-28 14:48:29.957036455 +0000 UTC m=+1779.758578671"
Jul 28 14:48:30.828153 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:30.828113    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:30.833306 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:30.833282    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:30.938925 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:30.938898    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:30.939990 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:30.939969    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:31.370615 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:31.370581    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"]
Jul 28 14:48:31.370991 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:31.370976    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea36761-2127-46f2-8362-7420e680d39c" containerName="istio-proxy"
Jul 28 14:48:31.370991 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:31.370992    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea36761-2127-46f2-8362-7420e680d39c" containerName="istio-proxy"
Jul 28 14:48:31.371119 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:31.371067    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ea36761-2127-46f2-8362-7420e680d39c" containerName="istio-proxy"
Jul 28 14:48:31.374245 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:31.374224    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"
Jul 28 14:48:31.376990 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:31.376963    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-e9a43c4a-serving-cert\""
Jul 28 14:48:31.385659 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:31.385633    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"]
Jul 28 14:48:31.434616 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:31.434578    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7224s\" (UniqueName: \"kubernetes.io/projected/a777ab8b-0e04-415b-8497-2a3255878494-kube-api-access-7224s\") pod \"maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf\" (UID: \"a777ab8b-0e04-415b-8497-2a3255878494\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"
Jul 28 14:48:31.434616 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:31.434619    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a777ab8b-0e04-415b-8497-2a3255878494-openshift-service-ca\") pod \"maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf\" (UID: \"a777ab8b-0e04-415b-8497-2a3255878494\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"
Jul 28 14:48:31.434839 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:31.434647    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a777ab8b-0e04-415b-8497-2a3255878494-maas-api-tls\") pod \"maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf\" (UID: \"a777ab8b-0e04-415b-8497-2a3255878494\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"
Jul 28 14:48:31.536053 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:31.536011    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7224s\" (UniqueName: \"kubernetes.io/projected/a777ab8b-0e04-415b-8497-2a3255878494-kube-api-access-7224s\") pod \"maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf\" (UID: \"a777ab8b-0e04-415b-8497-2a3255878494\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"
Jul 28 14:48:31.536277 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:31.536071    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a777ab8b-0e04-415b-8497-2a3255878494-openshift-service-ca\") pod \"maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf\" (UID: \"a777ab8b-0e04-415b-8497-2a3255878494\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"
Jul 28 14:48:31.536277 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:31.536142    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a777ab8b-0e04-415b-8497-2a3255878494-maas-api-tls\") pod \"maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf\" (UID: \"a777ab8b-0e04-415b-8497-2a3255878494\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"
Jul 28 14:48:31.537029 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:31.536984    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a777ab8b-0e04-415b-8497-2a3255878494-openshift-service-ca\") pod \"maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf\" (UID: \"a777ab8b-0e04-415b-8497-2a3255878494\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"
Jul 28 14:48:31.539504 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:31.539478    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a777ab8b-0e04-415b-8497-2a3255878494-maas-api-tls\") pod \"maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf\" (UID: \"a777ab8b-0e04-415b-8497-2a3255878494\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"
Jul 28 14:48:31.548859 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:31.548822    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7224s\" (UniqueName: \"kubernetes.io/projected/a777ab8b-0e04-415b-8497-2a3255878494-kube-api-access-7224s\") pod \"maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf\" (UID: \"a777ab8b-0e04-415b-8497-2a3255878494\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"
Jul 28 14:48:31.686365 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:31.686264    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"
Jul 28 14:48:31.822812 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:31.822630    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"]
Jul 28 14:48:31.825159 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:48:31.825126    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda777ab8b_0e04_415b_8497_2a3255878494.slice/crio-6801d0569f025f53680efec075a57bb030b4b6fdae3266489432296b7487aa86 WatchSource:0}: Error finding container 6801d0569f025f53680efec075a57bb030b4b6fdae3266489432296b7487aa86: Status 404 returned error can't find the container with id 6801d0569f025f53680efec075a57bb030b4b6fdae3266489432296b7487aa86
Jul 28 14:48:31.946480 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:31.946379    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf" event={"ID":"a777ab8b-0e04-415b-8497-2a3255878494","Type":"ContainerStarted","Data":"6801d0569f025f53680efec075a57bb030b4b6fdae3266489432296b7487aa86"}
Jul 28 14:48:32.137026 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.136993    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4"]
Jul 28 14:48:32.141863 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.141841    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4"
Jul 28 14:48:32.144680 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.144658    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-e9a43c4a\""
Jul 28 14:48:32.144828 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.144810    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-e9a43c4a-dockercfg-wznfz\""
Jul 28 14:48:32.159567 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.159545    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4"]
Jul 28 14:48:32.209395 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.209361    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh"]
Jul 28 14:48:32.214647 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.214626    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh"
Jul 28 14:48:32.227992 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.227971    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh"]
Jul 28 14:48:32.243478 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.242307    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjnmw\" (UniqueName: \"kubernetes.io/projected/ff4ef85e-673e-4eac-835f-6f68274ed677-kube-api-access-pjnmw\") pod \"payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4\" (UID: \"ff4ef85e-673e-4eac-835f-6f68274ed677\") " pod="openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4"
Jul 28 14:48:32.243478 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.242380    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff4ef85e-673e-4eac-835f-6f68274ed677-plugins-config-volume\") pod \"payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4\" (UID: \"ff4ef85e-673e-4eac-835f-6f68274ed677\") " pod="openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4"
Jul 28 14:48:32.243478 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.242442    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b54e02b2-c7b7-47c6-9780-daa2786ebd65-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh\" (UID: \"b54e02b2-c7b7-47c6-9780-daa2786ebd65\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh"
Jul 28 14:48:32.243478 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.242465    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq58r\" (UniqueName: \"kubernetes.io/projected/b54e02b2-c7b7-47c6-9780-daa2786ebd65-kube-api-access-bq58r\") pod \"payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh\" (UID: \"b54e02b2-c7b7-47c6-9780-daa2786ebd65\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh"
Jul 28 14:48:32.343179 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.343138    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjnmw\" (UniqueName: \"kubernetes.io/projected/ff4ef85e-673e-4eac-835f-6f68274ed677-kube-api-access-pjnmw\") pod \"payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4\" (UID: \"ff4ef85e-673e-4eac-835f-6f68274ed677\") " pod="openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4"
Jul 28 14:48:32.343350 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.343196    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff4ef85e-673e-4eac-835f-6f68274ed677-plugins-config-volume\") pod \"payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4\" (UID: \"ff4ef85e-673e-4eac-835f-6f68274ed677\") " pod="openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4"
Jul 28 14:48:32.343350 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.343247    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b54e02b2-c7b7-47c6-9780-daa2786ebd65-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh\" (UID: \"b54e02b2-c7b7-47c6-9780-daa2786ebd65\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh"
Jul 28 14:48:32.343350 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.343299    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq58r\" (UniqueName: \"kubernetes.io/projected/b54e02b2-c7b7-47c6-9780-daa2786ebd65-kube-api-access-bq58r\") pod \"payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh\" (UID: \"b54e02b2-c7b7-47c6-9780-daa2786ebd65\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh"
Jul 28 14:48:32.343932 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.343915    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b54e02b2-c7b7-47c6-9780-daa2786ebd65-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh\" (UID: \"b54e02b2-c7b7-47c6-9780-daa2786ebd65\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh"
Jul 28 14:48:32.343967 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.343937    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff4ef85e-673e-4eac-835f-6f68274ed677-plugins-config-volume\") pod \"payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4\" (UID: \"ff4ef85e-673e-4eac-835f-6f68274ed677\") " pod="openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4"
Jul 28 14:48:32.354669 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.354640    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq58r\" (UniqueName: \"kubernetes.io/projected/b54e02b2-c7b7-47c6-9780-daa2786ebd65-kube-api-access-bq58r\") pod \"payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh\" (UID: \"b54e02b2-c7b7-47c6-9780-daa2786ebd65\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh"
Jul 28 14:48:32.355823 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.355800    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjnmw\" (UniqueName: \"kubernetes.io/projected/ff4ef85e-673e-4eac-835f-6f68274ed677-kube-api-access-pjnmw\") pod \"payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4\" (UID: \"ff4ef85e-673e-4eac-835f-6f68274ed677\") " pod="openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4"
Jul 28 14:48:32.452210 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.452172    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4"
Jul 28 14:48:32.525648 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.525619    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh"
Jul 28 14:48:32.600431 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.600401    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4"]
Jul 28 14:48:32.602058 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:48:32.602032    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff4ef85e_673e_4eac_835f_6f68274ed677.slice/crio-2b58bbf9085ac6d637a24561ae5c22e6ae361d724079ab6bc086c0dbbae0259a WatchSource:0}: Error finding container 2b58bbf9085ac6d637a24561ae5c22e6ae361d724079ab6bc086c0dbbae0259a: Status 404 returned error can't find the container with id 2b58bbf9085ac6d637a24561ae5c22e6ae361d724079ab6bc086c0dbbae0259a
Jul 28 14:48:32.888897 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.888869    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh"]
Jul 28 14:48:32.891064 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:48:32.891037    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb54e02b2_c7b7_47c6_9780_daa2786ebd65.slice/crio-bfed9b34de50eebbb06f8752922e9ad62471d32be959c515c54d01feed56734d WatchSource:0}: Error finding container bfed9b34de50eebbb06f8752922e9ad62471d32be959c515c54d01feed56734d: Status 404 returned error can't find the container with id bfed9b34de50eebbb06f8752922e9ad62471d32be959c515c54d01feed56734d
Jul 28 14:48:32.953259 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.953227    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf" event={"ID":"a777ab8b-0e04-415b-8497-2a3255878494","Type":"ContainerStarted","Data":"730858d56d26a25bb76bb453d01f225a3c3d388702504c352664e8e441c68589"}
Jul 28 14:48:32.953634 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.953301    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"
Jul 28 14:48:32.954937 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.954909    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh" event={"ID":"b54e02b2-c7b7-47c6-9780-daa2786ebd65","Type":"ContainerStarted","Data":"bfed9b34de50eebbb06f8752922e9ad62471d32be959c515c54d01feed56734d"}
Jul 28 14:48:32.965318 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.965289    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4" event={"ID":"ff4ef85e-673e-4eac-835f-6f68274ed677","Type":"ContainerStarted","Data":"9c173def312cbdb5401be978177d67b55fd8cdb9531aaa5a4c63d12986ebef54"}
Jul 28 14:48:32.965454 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.965384    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4"
Jul 28 14:48:32.965454 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.965409    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4" event={"ID":"ff4ef85e-673e-4eac-835f-6f68274ed677","Type":"ContainerStarted","Data":"2b58bbf9085ac6d637a24561ae5c22e6ae361d724079ab6bc086c0dbbae0259a"}
Jul 28 14:48:32.981178 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:32.981130    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf" podStartSLOduration=1.626080828 podStartE2EDuration="1.981113838s" totalImagesPullingTime="355.03301ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:48:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:48:32.9790341 +0000 UTC m=+1782.780576318" watchObservedRunningTime="2026-07-28 14:48:32.981113838 +0000 UTC m=+1782.782656049"
Jul 28 14:48:33.000314 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:33.000253    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4" podStartSLOduration=1.000233813 podStartE2EDuration="1.000233813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:48:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:48:32.998935137 +0000 UTC m=+1782.800477354" watchObservedRunningTime="2026-07-28 14:48:33.000233813 +0000 UTC m=+1782.801776029"
Jul 28 14:48:33.971898 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:33.971851    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh" event={"ID":"b54e02b2-c7b7-47c6-9780-daa2786ebd65","Type":"ContainerStarted","Data":"72cfd93630b50af727449d715cd38f8ee37cdb12a5bd688ed5b3736b6406dc69"}
Jul 28 14:48:33.991434 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:33.991365    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh" podStartSLOduration=1.991344476 podStartE2EDuration="1.991344476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:48:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:48:33.990022472 +0000 UTC m=+1783.791564687" watchObservedRunningTime="2026-07-28 14:48:33.991344476 +0000 UTC m=+1783.792886694"
Jul 28 14:48:34.977338 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:34.977299    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh"
Jul 28 14:48:38.977570 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:38.977536    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"
Jul 28 14:48:43.974288 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:43.974253    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4"
Jul 28 14:48:44.026520 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:44.026489    2584 generic.go:356] "Generic (PLEG): container finished" podID="dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" containerID="4329e1bd542d7801470afc73d2bebaacc2ceb15205b4d50f69f0300285d54ac7" exitCode=0
Jul 28 14:48:44.026678 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:44.026570    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9" event={"ID":"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6","Type":"ContainerDied","Data":"4329e1bd542d7801470afc73d2bebaacc2ceb15205b4d50f69f0300285d54ac7"}
Jul 28 14:48:44.027058 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:44.027041    2584 scope.go:117] "RemoveContainer" containerID="4329e1bd542d7801470afc73d2bebaacc2ceb15205b4d50f69f0300285d54ac7"
Jul 28 14:48:44.028227 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:44.028198    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:48:44.028303 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:44.028278    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:48:44.028342 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:44.028329    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:48:45.052917 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:45.052856    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9" event={"ID":"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6","Type":"ContainerStarted","Data":"f41a126f8b0298420d5785cd0b1b27efb55d7a2d7e99a9d4a6bcd36629aab5e7"}
Jul 28 14:48:45.828262 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:45.828222    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:45.829931 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:45.829898    2584 patch_prober.go:28] interesting pod/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 28 14:48:45.830177 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:45.830149    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9" podUID="dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 28 14:48:45.939075 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:45.939028    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:45.985164 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:45.985135    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh"
Jul 28 14:48:46.828497 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:46.828455    2584 patch_prober.go:28] interesting pod/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 28 14:48:46.828926 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:46.828539    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9" podUID="dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 28 14:48:47.828088 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:47.828050    2584 patch_prober.go:28] interesting pod/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 28 14:48:47.828311 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:47.828138    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9" podUID="dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 28 14:48:48.831986 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:48.831955    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:48:48.832876 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:48:48.832856    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:49:02.144143 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.144110    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"]
Jul 28 14:49:02.144607 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.144344    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf" podUID="a777ab8b-0e04-415b-8497-2a3255878494" containerName="maas-api" containerID="cri-o://730858d56d26a25bb76bb453d01f225a3c3d388702504c352664e8e441c68589" gracePeriod=30
Jul 28 14:49:02.197436 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.197398    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4"]
Jul 28 14:49:02.197643 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.197608    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4" podUID="ff4ef85e-673e-4eac-835f-6f68274ed677" containerName="payload-processing" containerID="cri-o://9c173def312cbdb5401be978177d67b55fd8cdb9531aaa5a4c63d12986ebef54" gracePeriod=30
Jul 28 14:49:02.206203 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.206173    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh"]
Jul 28 14:49:02.206476 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.206429    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh" podUID="b54e02b2-c7b7-47c6-9780-daa2786ebd65" containerName="payload-pre-processing" containerID="cri-o://72cfd93630b50af727449d715cd38f8ee37cdb12a5bd688ed5b3736b6406dc69" gracePeriod=30
Jul 28 14:49:02.502369 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.502334    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"
Jul 28 14:49:02.505503 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.505433    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4"
Jul 28 14:49:02.537988 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.537885    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"]
Jul 28 14:49:02.538518 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.538339    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9" podUID="dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" containerName="istio-proxy" containerID="cri-o://f41a126f8b0298420d5785cd0b1b27efb55d7a2d7e99a9d4a6bcd36629aab5e7" gracePeriod=30
Jul 28 14:49:02.544810 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.544786    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh"
Jul 28 14:49:02.636720 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.636680    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a777ab8b-0e04-415b-8497-2a3255878494-maas-api-tls\") pod \"a777ab8b-0e04-415b-8497-2a3255878494\" (UID: \"a777ab8b-0e04-415b-8497-2a3255878494\") "
Jul 28 14:49:02.636946 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.636739    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7224s\" (UniqueName: \"kubernetes.io/projected/a777ab8b-0e04-415b-8497-2a3255878494-kube-api-access-7224s\") pod \"a777ab8b-0e04-415b-8497-2a3255878494\" (UID: \"a777ab8b-0e04-415b-8497-2a3255878494\") "
Jul 28 14:49:02.636946 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.636786    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b54e02b2-c7b7-47c6-9780-daa2786ebd65-plugins-config-volume\") pod \"b54e02b2-c7b7-47c6-9780-daa2786ebd65\" (UID: \"b54e02b2-c7b7-47c6-9780-daa2786ebd65\") "
Jul 28 14:49:02.636946 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.636831    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjnmw\" (UniqueName: \"kubernetes.io/projected/ff4ef85e-673e-4eac-835f-6f68274ed677-kube-api-access-pjnmw\") pod \"ff4ef85e-673e-4eac-835f-6f68274ed677\" (UID: \"ff4ef85e-673e-4eac-835f-6f68274ed677\") "
Jul 28 14:49:02.636946 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.636874    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff4ef85e-673e-4eac-835f-6f68274ed677-plugins-config-volume\") pod \"ff4ef85e-673e-4eac-835f-6f68274ed677\" (UID: \"ff4ef85e-673e-4eac-835f-6f68274ed677\") "
Jul 28 14:49:02.637869 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.636929    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a777ab8b-0e04-415b-8497-2a3255878494-openshift-service-ca\") pod \"a777ab8b-0e04-415b-8497-2a3255878494\" (UID: \"a777ab8b-0e04-415b-8497-2a3255878494\") "
Jul 28 14:49:02.637993 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.637943    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bq58r\" (UniqueName: \"kubernetes.io/projected/b54e02b2-c7b7-47c6-9780-daa2786ebd65-kube-api-access-bq58r\") pod \"b54e02b2-c7b7-47c6-9780-daa2786ebd65\" (UID: \"b54e02b2-c7b7-47c6-9780-daa2786ebd65\") "
Jul 28 14:49:02.637993 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.637506    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a777ab8b-0e04-415b-8497-2a3255878494-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a777ab8b-0e04-415b-8497-2a3255878494" (UID: "a777ab8b-0e04-415b-8497-2a3255878494"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:49:02.638136 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.637629    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b54e02b2-c7b7-47c6-9780-daa2786ebd65-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b54e02b2-c7b7-47c6-9780-daa2786ebd65" (UID: "b54e02b2-c7b7-47c6-9780-daa2786ebd65"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:49:02.638136 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.637809    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff4ef85e-673e-4eac-835f-6f68274ed677-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ff4ef85e-673e-4eac-835f-6f68274ed677" (UID: "ff4ef85e-673e-4eac-835f-6f68274ed677"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:49:02.638538 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.638497    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b54e02b2-c7b7-47c6-9780-daa2786ebd65-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:49:02.638634 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.638546    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff4ef85e-673e-4eac-835f-6f68274ed677-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:49:02.638634 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.638562    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a777ab8b-0e04-415b-8497-2a3255878494-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:49:02.639959 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.639930    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a777ab8b-0e04-415b-8497-2a3255878494-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a777ab8b-0e04-415b-8497-2a3255878494" (UID: "a777ab8b-0e04-415b-8497-2a3255878494"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:49:02.640388 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.640359    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a777ab8b-0e04-415b-8497-2a3255878494-kube-api-access-7224s" (OuterVolumeSpecName: "kube-api-access-7224s") pod "a777ab8b-0e04-415b-8497-2a3255878494" (UID: "a777ab8b-0e04-415b-8497-2a3255878494"). InnerVolumeSpecName "kube-api-access-7224s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:49:02.641289 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.641267    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff4ef85e-673e-4eac-835f-6f68274ed677-kube-api-access-pjnmw" (OuterVolumeSpecName: "kube-api-access-pjnmw") pod "ff4ef85e-673e-4eac-835f-6f68274ed677" (UID: "ff4ef85e-673e-4eac-835f-6f68274ed677"). InnerVolumeSpecName "kube-api-access-pjnmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:49:02.641547 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.641530    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b54e02b2-c7b7-47c6-9780-daa2786ebd65-kube-api-access-bq58r" (OuterVolumeSpecName: "kube-api-access-bq58r") pod "b54e02b2-c7b7-47c6-9780-daa2786ebd65" (UID: "b54e02b2-c7b7-47c6-9780-daa2786ebd65"). InnerVolumeSpecName "kube-api-access-bq58r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:49:02.739559 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.739474    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjnmw\" (UniqueName: \"kubernetes.io/projected/ff4ef85e-673e-4eac-835f-6f68274ed677-kube-api-access-pjnmw\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:49:02.739559 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.739503    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bq58r\" (UniqueName: \"kubernetes.io/projected/b54e02b2-c7b7-47c6-9780-daa2786ebd65-kube-api-access-bq58r\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:49:02.739559 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.739514    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a777ab8b-0e04-415b-8497-2a3255878494-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:49:02.739559 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:02.739523    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7224s\" (UniqueName: \"kubernetes.io/projected/a777ab8b-0e04-415b-8497-2a3255878494-kube-api-access-7224s\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:49:03.143523 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.143488    2584 generic.go:356] "Generic (PLEG): container finished" podID="b54e02b2-c7b7-47c6-9780-daa2786ebd65" containerID="72cfd93630b50af727449d715cd38f8ee37cdb12a5bd688ed5b3736b6406dc69" exitCode=0
Jul 28 14:49:03.143734 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.143528    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh"
Jul 28 14:49:03.143734 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.143572    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh" event={"ID":"b54e02b2-c7b7-47c6-9780-daa2786ebd65","Type":"ContainerDied","Data":"72cfd93630b50af727449d715cd38f8ee37cdb12a5bd688ed5b3736b6406dc69"}
Jul 28 14:49:03.143734 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.143630    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh" event={"ID":"b54e02b2-c7b7-47c6-9780-daa2786ebd65","Type":"ContainerDied","Data":"bfed9b34de50eebbb06f8752922e9ad62471d32be959c515c54d01feed56734d"}
Jul 28 14:49:03.143734 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.143665    2584 scope.go:117] "RemoveContainer" containerID="72cfd93630b50af727449d715cd38f8ee37cdb12a5bd688ed5b3736b6406dc69"
Jul 28 14:49:03.145059 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.145039    2584 generic.go:356] "Generic (PLEG): container finished" podID="ff4ef85e-673e-4eac-835f-6f68274ed677" containerID="9c173def312cbdb5401be978177d67b55fd8cdb9531aaa5a4c63d12986ebef54" exitCode=0
Jul 28 14:49:03.145459 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.145121    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4"
Jul 28 14:49:03.145459 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.145131    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4" event={"ID":"ff4ef85e-673e-4eac-835f-6f68274ed677","Type":"ContainerDied","Data":"9c173def312cbdb5401be978177d67b55fd8cdb9531aaa5a4c63d12986ebef54"}
Jul 28 14:49:03.145459 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.145176    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4" event={"ID":"ff4ef85e-673e-4eac-835f-6f68274ed677","Type":"ContainerDied","Data":"2b58bbf9085ac6d637a24561ae5c22e6ae361d724079ab6bc086c0dbbae0259a"}
Jul 28 14:49:03.146802 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.146777    2584 generic.go:356] "Generic (PLEG): container finished" podID="a777ab8b-0e04-415b-8497-2a3255878494" containerID="730858d56d26a25bb76bb453d01f225a3c3d388702504c352664e8e441c68589" exitCode=0
Jul 28 14:49:03.146918 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.146824    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"
Jul 28 14:49:03.146918 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.146845    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf" event={"ID":"a777ab8b-0e04-415b-8497-2a3255878494","Type":"ContainerDied","Data":"730858d56d26a25bb76bb453d01f225a3c3d388702504c352664e8e441c68589"}
Jul 28 14:49:03.146918 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.146886    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf" event={"ID":"a777ab8b-0e04-415b-8497-2a3255878494","Type":"ContainerDied","Data":"6801d0569f025f53680efec075a57bb030b4b6fdae3266489432296b7487aa86"}
Jul 28 14:49:03.154119 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.153906    2584 scope.go:117] "RemoveContainer" containerID="72cfd93630b50af727449d715cd38f8ee37cdb12a5bd688ed5b3736b6406dc69"
Jul 28 14:49:03.154236 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:49:03.154218    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72cfd93630b50af727449d715cd38f8ee37cdb12a5bd688ed5b3736b6406dc69\": container with ID starting with 72cfd93630b50af727449d715cd38f8ee37cdb12a5bd688ed5b3736b6406dc69 not found: ID does not exist" containerID="72cfd93630b50af727449d715cd38f8ee37cdb12a5bd688ed5b3736b6406dc69"
Jul 28 14:49:03.154277 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.154244    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72cfd93630b50af727449d715cd38f8ee37cdb12a5bd688ed5b3736b6406dc69"} err="failed to get container status \"72cfd93630b50af727449d715cd38f8ee37cdb12a5bd688ed5b3736b6406dc69\": rpc error: code = NotFound desc = could not find container \"72cfd93630b50af727449d715cd38f8ee37cdb12a5bd688ed5b3736b6406dc69\": container with ID starting with 72cfd93630b50af727449d715cd38f8ee37cdb12a5bd688ed5b3736b6406dc69 not found: ID does not exist"
Jul 28 14:49:03.154277 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.154262    2584 scope.go:117] "RemoveContainer" containerID="9c173def312cbdb5401be978177d67b55fd8cdb9531aaa5a4c63d12986ebef54"
Jul 28 14:49:03.167140 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.167122    2584 scope.go:117] "RemoveContainer" containerID="9c173def312cbdb5401be978177d67b55fd8cdb9531aaa5a4c63d12986ebef54"
Jul 28 14:49:03.167497 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:49:03.167466    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c173def312cbdb5401be978177d67b55fd8cdb9531aaa5a4c63d12986ebef54\": container with ID starting with 9c173def312cbdb5401be978177d67b55fd8cdb9531aaa5a4c63d12986ebef54 not found: ID does not exist" containerID="9c173def312cbdb5401be978177d67b55fd8cdb9531aaa5a4c63d12986ebef54"
Jul 28 14:49:03.167616 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.167500    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c173def312cbdb5401be978177d67b55fd8cdb9531aaa5a4c63d12986ebef54"} err="failed to get container status \"9c173def312cbdb5401be978177d67b55fd8cdb9531aaa5a4c63d12986ebef54\": rpc error: code = NotFound desc = could not find container \"9c173def312cbdb5401be978177d67b55fd8cdb9531aaa5a4c63d12986ebef54\": container with ID starting with 9c173def312cbdb5401be978177d67b55fd8cdb9531aaa5a4c63d12986ebef54 not found: ID does not exist"
Jul 28 14:49:03.167616 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.167520    2584 scope.go:117] "RemoveContainer" containerID="730858d56d26a25bb76bb453d01f225a3c3d388702504c352664e8e441c68589"
Jul 28 14:49:03.169653 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.169629    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"]
Jul 28 14:49:03.174698 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.174679    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e9a43c4a-67d5bdd677-dzdlf"]
Jul 28 14:49:03.175332 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.175315    2584 scope.go:117] "RemoveContainer" containerID="730858d56d26a25bb76bb453d01f225a3c3d388702504c352664e8e441c68589"
Jul 28 14:49:03.175615 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:49:03.175579    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"730858d56d26a25bb76bb453d01f225a3c3d388702504c352664e8e441c68589\": container with ID starting with 730858d56d26a25bb76bb453d01f225a3c3d388702504c352664e8e441c68589 not found: ID does not exist" containerID="730858d56d26a25bb76bb453d01f225a3c3d388702504c352664e8e441c68589"
Jul 28 14:49:03.175615 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.175606    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"730858d56d26a25bb76bb453d01f225a3c3d388702504c352664e8e441c68589"} err="failed to get container status \"730858d56d26a25bb76bb453d01f225a3c3d388702504c352664e8e441c68589\": rpc error: code = NotFound desc = could not find container \"730858d56d26a25bb76bb453d01f225a3c3d388702504c352664e8e441c68589\": container with ID starting with 730858d56d26a25bb76bb453d01f225a3c3d388702504c352664e8e441c68589 not found: ID does not exist"
Jul 28 14:49:03.189661 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.189639    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4"]
Jul 28 14:49:03.198818 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.198796    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e9a43c4a-7896df8774-f9hw4"]
Jul 28 14:49:03.210714 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.210688    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh"]
Jul 28 14:49:03.216320 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.216294    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-e9a43c4a-6b7d9c9784-brgdh"]
Jul 28 14:49:03.835146 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.835088    2584 patch_prober.go:28] interesting pod/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 14:49:03.835357 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:03.835194    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9" podUID="dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 14:49:04.796248 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:04.796220    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a777ab8b-0e04-415b-8497-2a3255878494" path="/var/lib/kubelet/pods/a777ab8b-0e04-415b-8497-2a3255878494/volumes"
Jul 28 14:49:04.796632 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:04.796608    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b54e02b2-c7b7-47c6-9780-daa2786ebd65" path="/var/lib/kubelet/pods/b54e02b2-c7b7-47c6-9780-daa2786ebd65/volumes"
Jul 28 14:49:04.796958 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:04.796945    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff4ef85e-673e-4eac-835f-6f68274ed677" path="/var/lib/kubelet/pods/ff4ef85e-673e-4eac-835f-6f68274ed677/volumes"
Jul 28 14:49:07.791832 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.791808    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:49:07.885732 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.885698    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-workload-certs\") pod \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") "
Jul 28 14:49:07.885732 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.885733    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-workload-socket\") pod \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") "
Jul 28 14:49:07.885984 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.885770    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-token\") pod \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") "
Jul 28 14:49:07.885984 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.885829    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-data\") pod \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") "
Jul 28 14:49:07.885984 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.885855    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-podinfo\") pod \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") "
Jul 28 14:49:07.885984 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.885909    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-credential-socket\") pod \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") "
Jul 28 14:49:07.885984 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.885936    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wntv6\" (UniqueName: \"kubernetes.io/projected/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-kube-api-access-wntv6\") pod \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") "
Jul 28 14:49:07.885984 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.885960    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istiod-ca-cert\") pod \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") "
Jul 28 14:49:07.886346 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.886012    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-envoy\") pod \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\" (UID: \"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6\") "
Jul 28 14:49:07.886346 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.885990    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" (UID: "dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:49:07.886346 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.886051    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" (UID: "dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:49:07.886346 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.886315    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-workload-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:49:07.886346 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.886341    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-workload-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:49:07.886557 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.886370    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" (UID: "dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:49:07.886557 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.886540    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" (UID: "dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:49:07.886764 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.886742    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-data" (OuterVolumeSpecName: "istio-data") pod "dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" (UID: "dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:49:07.888205 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.888175    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-kube-api-access-wntv6" (OuterVolumeSpecName: "kube-api-access-wntv6") pod "dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" (UID: "dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6"). InnerVolumeSpecName "kube-api-access-wntv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:49:07.888318 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.888208    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" (UID: "dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:49:07.888442 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.888422    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-token" (OuterVolumeSpecName: "istio-token") pod "dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" (UID: "dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:49:07.888547 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.888531    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" (UID: "dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 14:49:07.987182 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.987143    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-token\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:49:07.987182 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.987177    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-data\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:49:07.987182 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.987186    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-podinfo\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:49:07.987429 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.987196    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-credential-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:49:07.987429 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.987207    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wntv6\" (UniqueName: \"kubernetes.io/projected/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-kube-api-access-wntv6\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:49:07.987429 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.987216    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istiod-ca-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:49:07.987429 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:07.987224    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6-istio-envoy\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:49:08.179010 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:08.178919    2584 generic.go:356] "Generic (PLEG): container finished" podID="dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" containerID="f41a126f8b0298420d5785cd0b1b27efb55d7a2d7e99a9d4a6bcd36629aab5e7" exitCode=0
Jul 28 14:49:08.179010 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:08.178976    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"
Jul 28 14:49:08.179224 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:08.179000    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9" event={"ID":"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6","Type":"ContainerDied","Data":"f41a126f8b0298420d5785cd0b1b27efb55d7a2d7e99a9d4a6bcd36629aab5e7"}
Jul 28 14:49:08.179224 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:08.179060    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9" event={"ID":"dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6","Type":"ContainerDied","Data":"daa7f6e00edb10f857cc1a43548964a0f08d42fa7bfe7edd617fddd7ace32699"}
Jul 28 14:49:08.179224 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:08.179100    2584 scope.go:117] "RemoveContainer" containerID="f41a126f8b0298420d5785cd0b1b27efb55d7a2d7e99a9d4a6bcd36629aab5e7"
Jul 28 14:49:08.188441 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:08.188418    2584 scope.go:117] "RemoveContainer" containerID="4329e1bd542d7801470afc73d2bebaacc2ceb15205b4d50f69f0300285d54ac7"
Jul 28 14:49:08.196240 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:08.196216    2584 scope.go:117] "RemoveContainer" containerID="f41a126f8b0298420d5785cd0b1b27efb55d7a2d7e99a9d4a6bcd36629aab5e7"
Jul 28 14:49:08.196491 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:49:08.196470    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f41a126f8b0298420d5785cd0b1b27efb55d7a2d7e99a9d4a6bcd36629aab5e7\": container with ID starting with f41a126f8b0298420d5785cd0b1b27efb55d7a2d7e99a9d4a6bcd36629aab5e7 not found: ID does not exist" containerID="f41a126f8b0298420d5785cd0b1b27efb55d7a2d7e99a9d4a6bcd36629aab5e7"
Jul 28 14:49:08.196560 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:08.196499    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f41a126f8b0298420d5785cd0b1b27efb55d7a2d7e99a9d4a6bcd36629aab5e7"} err="failed to get container status \"f41a126f8b0298420d5785cd0b1b27efb55d7a2d7e99a9d4a6bcd36629aab5e7\": rpc error: code = NotFound desc = could not find container \"f41a126f8b0298420d5785cd0b1b27efb55d7a2d7e99a9d4a6bcd36629aab5e7\": container with ID starting with f41a126f8b0298420d5785cd0b1b27efb55d7a2d7e99a9d4a6bcd36629aab5e7 not found: ID does not exist"
Jul 28 14:49:08.196560 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:08.196517    2584 scope.go:117] "RemoveContainer" containerID="4329e1bd542d7801470afc73d2bebaacc2ceb15205b4d50f69f0300285d54ac7"
Jul 28 14:49:08.196765 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:49:08.196745    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4329e1bd542d7801470afc73d2bebaacc2ceb15205b4d50f69f0300285d54ac7\": container with ID starting with 4329e1bd542d7801470afc73d2bebaacc2ceb15205b4d50f69f0300285d54ac7 not found: ID does not exist" containerID="4329e1bd542d7801470afc73d2bebaacc2ceb15205b4d50f69f0300285d54ac7"
Jul 28 14:49:08.196808 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:08.196776    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4329e1bd542d7801470afc73d2bebaacc2ceb15205b4d50f69f0300285d54ac7"} err="failed to get container status \"4329e1bd542d7801470afc73d2bebaacc2ceb15205b4d50f69f0300285d54ac7\": rpc error: code = NotFound desc = could not find container \"4329e1bd542d7801470afc73d2bebaacc2ceb15205b4d50f69f0300285d54ac7\": container with ID starting with 4329e1bd542d7801470afc73d2bebaacc2ceb15205b4d50f69f0300285d54ac7 not found: ID does not exist"
Jul 28 14:49:08.208748 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:08.208690    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"]
Jul 28 14:49:08.210312 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:08.210290    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-e9a43c4a-openshift-default-7f5dfcbdb4-k5qh9"]
Jul 28 14:49:08.797391 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:49:08.797358    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" path="/var/lib/kubelet/pods/dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6/volumes"
Jul 28 14:57:58.394560 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.394471    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"]
Jul 28 14:57:58.396913 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.395066    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" containerName="istio-proxy"
Jul 28 14:57:58.396913 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.395087    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" containerName="istio-proxy"
Jul 28 14:57:58.396913 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.395147    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff4ef85e-673e-4eac-835f-6f68274ed677" containerName="payload-processing"
Jul 28 14:57:58.396913 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.395157    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff4ef85e-673e-4eac-835f-6f68274ed677" containerName="payload-processing"
Jul 28 14:57:58.396913 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.395169    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a777ab8b-0e04-415b-8497-2a3255878494" containerName="maas-api"
Jul 28 14:57:58.396913 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.395180    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a777ab8b-0e04-415b-8497-2a3255878494" containerName="maas-api"
Jul 28 14:57:58.396913 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.395209    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b54e02b2-c7b7-47c6-9780-daa2786ebd65" containerName="payload-pre-processing"
Jul 28 14:57:58.396913 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.395218    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b54e02b2-c7b7-47c6-9780-daa2786ebd65" containerName="payload-pre-processing"
Jul 28 14:57:58.396913 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.395314    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b54e02b2-c7b7-47c6-9780-daa2786ebd65" containerName="payload-pre-processing"
Jul 28 14:57:58.396913 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.395326    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff4ef85e-673e-4eac-835f-6f68274ed677" containerName="payload-processing"
Jul 28 14:57:58.396913 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.395338    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" containerName="istio-proxy"
Jul 28 14:57:58.396913 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.395351    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a777ab8b-0e04-415b-8497-2a3255878494" containerName="maas-api"
Jul 28 14:57:58.396913 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.395359    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" containerName="istio-proxy"
Jul 28 14:57:58.398613 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.398593    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"
Jul 28 14:57:58.401437 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.401414    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-8b4faed4-serving-cert\""
Jul 28 14:57:58.412368 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.412335    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"]
Jul 28 14:57:58.456307 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.456264    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-openshift-service-ca\") pod \"maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv\" (UID: \"7bc968ab-ab07-49bd-abb7-d1ce58284ea6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"
Jul 28 14:57:58.456307 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.456310    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-maas-api-tls\") pod \"maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv\" (UID: \"7bc968ab-ab07-49bd-abb7-d1ce58284ea6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"
Jul 28 14:57:58.456532 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.456331    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6bz9\" (UniqueName: \"kubernetes.io/projected/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-kube-api-access-r6bz9\") pod \"maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv\" (UID: \"7bc968ab-ab07-49bd-abb7-d1ce58284ea6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"
Jul 28 14:57:58.556993 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.556955    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-openshift-service-ca\") pod \"maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv\" (UID: \"7bc968ab-ab07-49bd-abb7-d1ce58284ea6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"
Jul 28 14:57:58.556993 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.556997    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-maas-api-tls\") pod \"maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv\" (UID: \"7bc968ab-ab07-49bd-abb7-d1ce58284ea6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"
Jul 28 14:57:58.557304 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.557018    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6bz9\" (UniqueName: \"kubernetes.io/projected/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-kube-api-access-r6bz9\") pod \"maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv\" (UID: \"7bc968ab-ab07-49bd-abb7-d1ce58284ea6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"
Jul 28 14:57:58.557304 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:57:58.557161    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-serving-cert: secret "maas-api-e2e-mt-8b4faed4-serving-cert" not found
Jul 28 14:57:58.557304 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:57:58.557241    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-maas-api-tls podName:7bc968ab-ab07-49bd-abb7-d1ce58284ea6 nodeName:}" failed. No retries permitted until 2026-07-28 14:57:59.057219625 +0000 UTC m=+2348.858761818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-maas-api-tls") pod "maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv" (UID: "7bc968ab-ab07-49bd-abb7-d1ce58284ea6") : secret "maas-api-e2e-mt-8b4faed4-serving-cert" not found
Jul 28 14:57:58.557633 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.557615    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-openshift-service-ca\") pod \"maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv\" (UID: \"7bc968ab-ab07-49bd-abb7-d1ce58284ea6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"
Jul 28 14:57:58.577656 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:58.577626    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6bz9\" (UniqueName: \"kubernetes.io/projected/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-kube-api-access-r6bz9\") pod \"maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv\" (UID: \"7bc968ab-ab07-49bd-abb7-d1ce58284ea6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"
Jul 28 14:57:59.062001 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.061965    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-maas-api-tls\") pod \"maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv\" (UID: \"7bc968ab-ab07-49bd-abb7-d1ce58284ea6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"
Jul 28 14:57:59.064368 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.064334    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b"]
Jul 28 14:57:59.064517 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.064497    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-maas-api-tls\") pod \"maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv\" (UID: \"7bc968ab-ab07-49bd-abb7-d1ce58284ea6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"
Jul 28 14:57:59.064718 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.064704    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" containerName="istio-proxy"
Jul 28 14:57:59.064761 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.064718    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcdb5eaa-8308-494c-a3d6-589e9cb4e2a6" containerName="istio-proxy"
Jul 28 14:57:59.067695 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.067680    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b"
Jul 28 14:57:59.070263 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.070236    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-8b4faed4\""
Jul 28 14:57:59.079494 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.079471    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b"]
Jul 28 14:57:59.121042 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.121008    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4"]
Jul 28 14:57:59.124534 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.124517    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4"
Jul 28 14:57:59.126847 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.126827    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-8b4faed4-dockercfg-hfcrz\""
Jul 28 14:57:59.138124 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.138078    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4"]
Jul 28 14:57:59.162690 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.162661    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7pjh\" (UniqueName: \"kubernetes.io/projected/126006e8-d01d-4bef-a87c-7a697f71e836-kube-api-access-l7pjh\") pod \"payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4\" (UID: \"126006e8-d01d-4bef-a87c-7a697f71e836\") " pod="openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4"
Jul 28 14:57:59.162856 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.162722    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/126006e8-d01d-4bef-a87c-7a697f71e836-plugins-config-volume\") pod \"payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4\" (UID: \"126006e8-d01d-4bef-a87c-7a697f71e836\") " pod="openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4"
Jul 28 14:57:59.162856 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.162777    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2caca922-1a67-41b4-9a55-06d3a5ea9f26-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b\" (UID: \"2caca922-1a67-41b4-9a55-06d3a5ea9f26\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b"
Jul 28 14:57:59.162856 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.162795    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7scl\" (UniqueName: \"kubernetes.io/projected/2caca922-1a67-41b4-9a55-06d3a5ea9f26-kube-api-access-g7scl\") pod \"payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b\" (UID: \"2caca922-1a67-41b4-9a55-06d3a5ea9f26\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b"
Jul 28 14:57:59.264296 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.264258    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/126006e8-d01d-4bef-a87c-7a697f71e836-plugins-config-volume\") pod \"payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4\" (UID: \"126006e8-d01d-4bef-a87c-7a697f71e836\") " pod="openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4"
Jul 28 14:57:59.264484 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.264307    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2caca922-1a67-41b4-9a55-06d3a5ea9f26-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b\" (UID: \"2caca922-1a67-41b4-9a55-06d3a5ea9f26\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b"
Jul 28 14:57:59.264484 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.264326    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7scl\" (UniqueName: \"kubernetes.io/projected/2caca922-1a67-41b4-9a55-06d3a5ea9f26-kube-api-access-g7scl\") pod \"payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b\" (UID: \"2caca922-1a67-41b4-9a55-06d3a5ea9f26\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b"
Jul 28 14:57:59.264484 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.264388    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7pjh\" (UniqueName: \"kubernetes.io/projected/126006e8-d01d-4bef-a87c-7a697f71e836-kube-api-access-l7pjh\") pod \"payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4\" (UID: \"126006e8-d01d-4bef-a87c-7a697f71e836\") " pod="openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4"
Jul 28 14:57:59.265018 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.264996    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2caca922-1a67-41b4-9a55-06d3a5ea9f26-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b\" (UID: \"2caca922-1a67-41b4-9a55-06d3a5ea9f26\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b"
Jul 28 14:57:59.265068 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.265027    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/126006e8-d01d-4bef-a87c-7a697f71e836-plugins-config-volume\") pod \"payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4\" (UID: \"126006e8-d01d-4bef-a87c-7a697f71e836\") " pod="openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4"
Jul 28 14:57:59.272670 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.272648    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7scl\" (UniqueName: \"kubernetes.io/projected/2caca922-1a67-41b4-9a55-06d3a5ea9f26-kube-api-access-g7scl\") pod \"payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b\" (UID: \"2caca922-1a67-41b4-9a55-06d3a5ea9f26\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b"
Jul 28 14:57:59.272802 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.272775    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7pjh\" (UniqueName: \"kubernetes.io/projected/126006e8-d01d-4bef-a87c-7a697f71e836-kube-api-access-l7pjh\") pod \"payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4\" (UID: \"126006e8-d01d-4bef-a87c-7a697f71e836\") " pod="openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4"
Jul 28 14:57:59.309579 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.309549    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"
Jul 28 14:57:59.382828 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.382796    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b"
Jul 28 14:57:59.434788 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.434704    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4"
Jul 28 14:57:59.448709 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.448678    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"]
Jul 28 14:57:59.453214 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:57:59.453185    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bc968ab_ab07_49bd_abb7_d1ce58284ea6.slice/crio-6c62d2465092379bde282f4976b7caec80eb2047f836ae456f06b3501230de52 WatchSource:0}: Error finding container 6c62d2465092379bde282f4976b7caec80eb2047f836ae456f06b3501230de52: Status 404 returned error can't find the container with id 6c62d2465092379bde282f4976b7caec80eb2047f836ae456f06b3501230de52
Jul 28 14:57:59.455900 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.455679    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 14:57:59.532254 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.532228    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b"]
Jul 28 14:57:59.534646 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:57:59.534617    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2caca922_1a67_41b4_9a55_06d3a5ea9f26.slice/crio-fae61bddec859c7825c802f3f6694c17ec24dfde7646d85a7015b70ed9b50bab WatchSource:0}: Error finding container fae61bddec859c7825c802f3f6694c17ec24dfde7646d85a7015b70ed9b50bab: Status 404 returned error can't find the container with id fae61bddec859c7825c802f3f6694c17ec24dfde7646d85a7015b70ed9b50bab
Jul 28 14:57:59.592511 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.592487    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4"]
Jul 28 14:57:59.594566 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:57:59.594535    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod126006e8_d01d_4bef_a87c_7a697f71e836.slice/crio-266b948de6c0fdc2950f2dd13026e1a06f03d3799b4b904eba193f18cb4729d7 WatchSource:0}: Error finding container 266b948de6c0fdc2950f2dd13026e1a06f03d3799b4b904eba193f18cb4729d7: Status 404 returned error can't find the container with id 266b948de6c0fdc2950f2dd13026e1a06f03d3799b4b904eba193f18cb4729d7
Jul 28 14:57:59.598806 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.598778    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b" event={"ID":"2caca922-1a67-41b4-9a55-06d3a5ea9f26","Type":"ContainerStarted","Data":"fae61bddec859c7825c802f3f6694c17ec24dfde7646d85a7015b70ed9b50bab"}
Jul 28 14:57:59.600149 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:57:59.600120    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv" event={"ID":"7bc968ab-ab07-49bd-abb7-d1ce58284ea6","Type":"ContainerStarted","Data":"6c62d2465092379bde282f4976b7caec80eb2047f836ae456f06b3501230de52"}
Jul 28 14:58:00.607266 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:00.607226    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv" event={"ID":"7bc968ab-ab07-49bd-abb7-d1ce58284ea6","Type":"ContainerStarted","Data":"746b359f22003918839095cbcb2fc878165a92bed4a7236015ca8b5f6d69c379"}
Jul 28 14:58:00.607734 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:00.607291    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"
Jul 28 14:58:00.608833 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:00.608806    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4" event={"ID":"126006e8-d01d-4bef-a87c-7a697f71e836","Type":"ContainerStarted","Data":"fe8890bfd7189a6f971ba24adca375c5341bf454e465d7c4125be60e3a2adab4"}
Jul 28 14:58:00.608978 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:00.608851    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4" event={"ID":"126006e8-d01d-4bef-a87c-7a697f71e836","Type":"ContainerStarted","Data":"266b948de6c0fdc2950f2dd13026e1a06f03d3799b4b904eba193f18cb4729d7"}
Jul 28 14:58:00.608978 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:00.608896    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4"
Jul 28 14:58:00.610223 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:00.610197    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b" event={"ID":"2caca922-1a67-41b4-9a55-06d3a5ea9f26","Type":"ContainerStarted","Data":"279269778fe03f1e70efabf5009aa69a462ed6044711648d99e87233b32856b6"}
Jul 28 14:58:00.610396 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:00.610384    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b"
Jul 28 14:58:00.630637 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:00.630562    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv" podStartSLOduration=2.170273635 podStartE2EDuration="2.630539832s" totalImagesPullingTime="460.266197ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:58:00.626448154 +0000 UTC m=+2350.427990369" watchObservedRunningTime="2026-07-28 14:58:00.630539832 +0000 UTC m=+2350.432082047"
Jul 28 14:58:00.649439 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:00.649385    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b" podStartSLOduration=1.64937091 podStartE2EDuration="1.64937091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:57:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:58:00.647378067 +0000 UTC m=+2350.448920296" watchObservedRunningTime="2026-07-28 14:58:00.64937091 +0000 UTC m=+2350.450913122"
Jul 28 14:58:00.665018 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:00.664964    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4" podStartSLOduration=1.66495115 podStartE2EDuration="1.66495115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:57:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:58:00.663601872 +0000 UTC m=+2350.465144087" watchObservedRunningTime="2026-07-28 14:58:00.66495115 +0000 UTC m=+2350.466493363"
Jul 28 14:58:06.622050 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:06.622018    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"
Jul 28 14:58:11.618211 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:11.618174    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b"
Jul 28 14:58:11.618712 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:11.618242    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4"
Jul 28 14:58:41.390042 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.390008    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"]
Jul 28 14:58:41.390669 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.390242    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv" podUID="7bc968ab-ab07-49bd-abb7-d1ce58284ea6" containerName="maas-api" containerID="cri-o://746b359f22003918839095cbcb2fc878165a92bed4a7236015ca8b5f6d69c379" gracePeriod=30
Jul 28 14:58:41.442905 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.442869    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4"]
Jul 28 14:58:41.443124 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.443083    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4" podUID="126006e8-d01d-4bef-a87c-7a697f71e836" containerName="payload-processing" containerID="cri-o://fe8890bfd7189a6f971ba24adca375c5341bf454e465d7c4125be60e3a2adab4" gracePeriod=30
Jul 28 14:58:41.449856 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.448971    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b"]
Jul 28 14:58:41.449856 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.449211    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b" podUID="2caca922-1a67-41b4-9a55-06d3a5ea9f26" containerName="payload-pre-processing" containerID="cri-o://279269778fe03f1e70efabf5009aa69a462ed6044711648d99e87233b32856b6" gracePeriod=30
Jul 28 14:58:41.620510 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.620424    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4" podUID="126006e8-d01d-4bef-a87c-7a697f71e836" containerName="payload-processing" probeResult="failure" output="dial tcp 10.134.0.60:9004: connect: connection refused"
Jul 28 14:58:41.620510 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.620454    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv" podUID="7bc968ab-ab07-49bd-abb7-d1ce58284ea6" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.58:8443/health\": dial tcp 10.134.0.58:8443: connect: connection refused"
Jul 28 14:58:41.620745 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.620424    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b" podUID="2caca922-1a67-41b4-9a55-06d3a5ea9f26" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.59:9004: connect: connection refused"
Jul 28 14:58:41.727422 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.727393    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"
Jul 28 14:58:41.744928 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.744844    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b"
Jul 28 14:58:41.756318 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.756283    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4"
Jul 28 14:58:41.768688 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.768662    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6bz9\" (UniqueName: \"kubernetes.io/projected/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-kube-api-access-r6bz9\") pod \"7bc968ab-ab07-49bd-abb7-d1ce58284ea6\" (UID: \"7bc968ab-ab07-49bd-abb7-d1ce58284ea6\") "
Jul 28 14:58:41.768833 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.768727    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-openshift-service-ca\") pod \"7bc968ab-ab07-49bd-abb7-d1ce58284ea6\" (UID: \"7bc968ab-ab07-49bd-abb7-d1ce58284ea6\") "
Jul 28 14:58:41.768833 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.768744    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2caca922-1a67-41b4-9a55-06d3a5ea9f26-plugins-config-volume\") pod \"2caca922-1a67-41b4-9a55-06d3a5ea9f26\" (UID: \"2caca922-1a67-41b4-9a55-06d3a5ea9f26\") "
Jul 28 14:58:41.768833 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.768800    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-maas-api-tls\") pod \"7bc968ab-ab07-49bd-abb7-d1ce58284ea6\" (UID: \"7bc968ab-ab07-49bd-abb7-d1ce58284ea6\") "
Jul 28 14:58:41.769027 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.768835    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7scl\" (UniqueName: \"kubernetes.io/projected/2caca922-1a67-41b4-9a55-06d3a5ea9f26-kube-api-access-g7scl\") pod \"2caca922-1a67-41b4-9a55-06d3a5ea9f26\" (UID: \"2caca922-1a67-41b4-9a55-06d3a5ea9f26\") "
Jul 28 14:58:41.769135 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.769106    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7bc968ab-ab07-49bd-abb7-d1ce58284ea6" (UID: "7bc968ab-ab07-49bd-abb7-d1ce58284ea6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:58:41.769251 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.769223    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2caca922-1a67-41b4-9a55-06d3a5ea9f26-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2caca922-1a67-41b4-9a55-06d3a5ea9f26" (UID: "2caca922-1a67-41b4-9a55-06d3a5ea9f26"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:58:41.770896 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.770872    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-kube-api-access-r6bz9" (OuterVolumeSpecName: "kube-api-access-r6bz9") pod "7bc968ab-ab07-49bd-abb7-d1ce58284ea6" (UID: "7bc968ab-ab07-49bd-abb7-d1ce58284ea6"). InnerVolumeSpecName "kube-api-access-r6bz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:58:41.770994 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.770975    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7bc968ab-ab07-49bd-abb7-d1ce58284ea6" (UID: "7bc968ab-ab07-49bd-abb7-d1ce58284ea6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:58:41.771066 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.771052    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2caca922-1a67-41b4-9a55-06d3a5ea9f26-kube-api-access-g7scl" (OuterVolumeSpecName: "kube-api-access-g7scl") pod "2caca922-1a67-41b4-9a55-06d3a5ea9f26" (UID: "2caca922-1a67-41b4-9a55-06d3a5ea9f26"). InnerVolumeSpecName "kube-api-access-g7scl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:58:41.809759 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.809719    2584 generic.go:356] "Generic (PLEG): container finished" podID="7bc968ab-ab07-49bd-abb7-d1ce58284ea6" containerID="746b359f22003918839095cbcb2fc878165a92bed4a7236015ca8b5f6d69c379" exitCode=0
Jul 28 14:58:41.809899 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.809765    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"
Jul 28 14:58:41.809899 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.809802    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv" event={"ID":"7bc968ab-ab07-49bd-abb7-d1ce58284ea6","Type":"ContainerDied","Data":"746b359f22003918839095cbcb2fc878165a92bed4a7236015ca8b5f6d69c379"}
Jul 28 14:58:41.809899 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.809854    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv" event={"ID":"7bc968ab-ab07-49bd-abb7-d1ce58284ea6","Type":"ContainerDied","Data":"6c62d2465092379bde282f4976b7caec80eb2047f836ae456f06b3501230de52"}
Jul 28 14:58:41.809899 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.809878    2584 scope.go:117] "RemoveContainer" containerID="746b359f22003918839095cbcb2fc878165a92bed4a7236015ca8b5f6d69c379"
Jul 28 14:58:41.811157 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.811132    2584 generic.go:356] "Generic (PLEG): container finished" podID="126006e8-d01d-4bef-a87c-7a697f71e836" containerID="fe8890bfd7189a6f971ba24adca375c5341bf454e465d7c4125be60e3a2adab4" exitCode=0
Jul 28 14:58:41.811269 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.811228    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4"
Jul 28 14:58:41.811338 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.811266    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4" event={"ID":"126006e8-d01d-4bef-a87c-7a697f71e836","Type":"ContainerDied","Data":"fe8890bfd7189a6f971ba24adca375c5341bf454e465d7c4125be60e3a2adab4"}
Jul 28 14:58:41.811338 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.811319    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4" event={"ID":"126006e8-d01d-4bef-a87c-7a697f71e836","Type":"ContainerDied","Data":"266b948de6c0fdc2950f2dd13026e1a06f03d3799b4b904eba193f18cb4729d7"}
Jul 28 14:58:41.812591 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.812568    2584 generic.go:356] "Generic (PLEG): container finished" podID="2caca922-1a67-41b4-9a55-06d3a5ea9f26" containerID="279269778fe03f1e70efabf5009aa69a462ed6044711648d99e87233b32856b6" exitCode=0
Jul 28 14:58:41.812726 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.812610    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b"
Jul 28 14:58:41.812726 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.812633    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b" event={"ID":"2caca922-1a67-41b4-9a55-06d3a5ea9f26","Type":"ContainerDied","Data":"279269778fe03f1e70efabf5009aa69a462ed6044711648d99e87233b32856b6"}
Jul 28 14:58:41.812726 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.812675    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b" event={"ID":"2caca922-1a67-41b4-9a55-06d3a5ea9f26","Type":"ContainerDied","Data":"fae61bddec859c7825c802f3f6694c17ec24dfde7646d85a7015b70ed9b50bab"}
Jul 28 14:58:41.821734 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.821655    2584 scope.go:117] "RemoveContainer" containerID="746b359f22003918839095cbcb2fc878165a92bed4a7236015ca8b5f6d69c379"
Jul 28 14:58:41.822634 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:58:41.822613    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"746b359f22003918839095cbcb2fc878165a92bed4a7236015ca8b5f6d69c379\": container with ID starting with 746b359f22003918839095cbcb2fc878165a92bed4a7236015ca8b5f6d69c379 not found: ID does not exist" containerID="746b359f22003918839095cbcb2fc878165a92bed4a7236015ca8b5f6d69c379"
Jul 28 14:58:41.822727 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.822643    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"746b359f22003918839095cbcb2fc878165a92bed4a7236015ca8b5f6d69c379"} err="failed to get container status \"746b359f22003918839095cbcb2fc878165a92bed4a7236015ca8b5f6d69c379\": rpc error: code = NotFound desc = could not find container \"746b359f22003918839095cbcb2fc878165a92bed4a7236015ca8b5f6d69c379\": container with ID starting with 746b359f22003918839095cbcb2fc878165a92bed4a7236015ca8b5f6d69c379 not found: ID does not exist"
Jul 28 14:58:41.822727 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.822661    2584 scope.go:117] "RemoveContainer" containerID="fe8890bfd7189a6f971ba24adca375c5341bf454e465d7c4125be60e3a2adab4"
Jul 28 14:58:41.830505 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.830489    2584 scope.go:117] "RemoveContainer" containerID="fe8890bfd7189a6f971ba24adca375c5341bf454e465d7c4125be60e3a2adab4"
Jul 28 14:58:41.830766 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:58:41.830745    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe8890bfd7189a6f971ba24adca375c5341bf454e465d7c4125be60e3a2adab4\": container with ID starting with fe8890bfd7189a6f971ba24adca375c5341bf454e465d7c4125be60e3a2adab4 not found: ID does not exist" containerID="fe8890bfd7189a6f971ba24adca375c5341bf454e465d7c4125be60e3a2adab4"
Jul 28 14:58:41.830861 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.830769    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe8890bfd7189a6f971ba24adca375c5341bf454e465d7c4125be60e3a2adab4"} err="failed to get container status \"fe8890bfd7189a6f971ba24adca375c5341bf454e465d7c4125be60e3a2adab4\": rpc error: code = NotFound desc = could not find container \"fe8890bfd7189a6f971ba24adca375c5341bf454e465d7c4125be60e3a2adab4\": container with ID starting with fe8890bfd7189a6f971ba24adca375c5341bf454e465d7c4125be60e3a2adab4 not found: ID does not exist"
Jul 28 14:58:41.830861 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.830788    2584 scope.go:117] "RemoveContainer" containerID="279269778fe03f1e70efabf5009aa69a462ed6044711648d99e87233b32856b6"
Jul 28 14:58:41.835841 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.835821    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"]
Jul 28 14:58:41.838448 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.838428    2584 scope.go:117] "RemoveContainer" containerID="279269778fe03f1e70efabf5009aa69a462ed6044711648d99e87233b32856b6"
Jul 28 14:58:41.838687 ip-10-0-130-66 kubenswrapper[2584]: E0728 14:58:41.838670    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"279269778fe03f1e70efabf5009aa69a462ed6044711648d99e87233b32856b6\": container with ID starting with 279269778fe03f1e70efabf5009aa69a462ed6044711648d99e87233b32856b6 not found: ID does not exist" containerID="279269778fe03f1e70efabf5009aa69a462ed6044711648d99e87233b32856b6"
Jul 28 14:58:41.838748 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.838694    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"279269778fe03f1e70efabf5009aa69a462ed6044711648d99e87233b32856b6"} err="failed to get container status \"279269778fe03f1e70efabf5009aa69a462ed6044711648d99e87233b32856b6\": rpc error: code = NotFound desc = could not find container \"279269778fe03f1e70efabf5009aa69a462ed6044711648d99e87233b32856b6\": container with ID starting with 279269778fe03f1e70efabf5009aa69a462ed6044711648d99e87233b32856b6 not found: ID does not exist"
Jul 28 14:58:41.841630 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.841603    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8b4faed4-85f67486ff-gw7kv"]
Jul 28 14:58:41.850898 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.850878    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b"]
Jul 28 14:58:41.856637 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.856617    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8b4faed4-6786f47bd4-4bg9b"]
Jul 28 14:58:41.870192 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.870174    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7pjh\" (UniqueName: \"kubernetes.io/projected/126006e8-d01d-4bef-a87c-7a697f71e836-kube-api-access-l7pjh\") pod \"126006e8-d01d-4bef-a87c-7a697f71e836\" (UID: \"126006e8-d01d-4bef-a87c-7a697f71e836\") "
Jul 28 14:58:41.870272 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.870258    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/126006e8-d01d-4bef-a87c-7a697f71e836-plugins-config-volume\") pod \"126006e8-d01d-4bef-a87c-7a697f71e836\" (UID: \"126006e8-d01d-4bef-a87c-7a697f71e836\") "
Jul 28 14:58:41.870546 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.870524    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:58:41.870645 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.870550    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7scl\" (UniqueName: \"kubernetes.io/projected/2caca922-1a67-41b4-9a55-06d3a5ea9f26-kube-api-access-g7scl\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:58:41.870645 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.870566    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6bz9\" (UniqueName: \"kubernetes.io/projected/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-kube-api-access-r6bz9\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:58:41.870645 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.870583    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bc968ab-ab07-49bd-abb7-d1ce58284ea6-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:58:41.870645 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.870596    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2caca922-1a67-41b4-9a55-06d3a5ea9f26-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:58:41.870645 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.870635    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/126006e8-d01d-4bef-a87c-7a697f71e836-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "126006e8-d01d-4bef-a87c-7a697f71e836" (UID: "126006e8-d01d-4bef-a87c-7a697f71e836"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:58:41.872087 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.872067    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/126006e8-d01d-4bef-a87c-7a697f71e836-kube-api-access-l7pjh" (OuterVolumeSpecName: "kube-api-access-l7pjh") pod "126006e8-d01d-4bef-a87c-7a697f71e836" (UID: "126006e8-d01d-4bef-a87c-7a697f71e836"). InnerVolumeSpecName "kube-api-access-l7pjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:58:41.971900 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.971798    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7pjh\" (UniqueName: \"kubernetes.io/projected/126006e8-d01d-4bef-a87c-7a697f71e836-kube-api-access-l7pjh\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:58:41.971900 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:41.971842    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/126006e8-d01d-4bef-a87c-7a697f71e836-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 14:58:42.133461 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:42.133425    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4"]
Jul 28 14:58:42.136861 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:42.136832    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8b4faed4-854585f7c4-lw2q4"]
Jul 28 14:58:42.796604 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:42.796525    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="126006e8-d01d-4bef-a87c-7a697f71e836" path="/var/lib/kubelet/pods/126006e8-d01d-4bef-a87c-7a697f71e836/volumes"
Jul 28 14:58:42.796948 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:42.796875    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2caca922-1a67-41b4-9a55-06d3a5ea9f26" path="/var/lib/kubelet/pods/2caca922-1a67-41b4-9a55-06d3a5ea9f26/volumes"
Jul 28 14:58:42.797233 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:58:42.797219    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bc968ab-ab07-49bd-abb7-d1ce58284ea6" path="/var/lib/kubelet/pods/7bc968ab-ab07-49bd-abb7-d1ce58284ea6/volumes"
Jul 28 14:59:08.658561 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.658524    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"]
Jul 28 14:59:08.659000 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.658977    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2caca922-1a67-41b4-9a55-06d3a5ea9f26" containerName="payload-pre-processing"
Jul 28 14:59:08.659000 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.658995    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2caca922-1a67-41b4-9a55-06d3a5ea9f26" containerName="payload-pre-processing"
Jul 28 14:59:08.659128 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.659021    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bc968ab-ab07-49bd-abb7-d1ce58284ea6" containerName="maas-api"
Jul 28 14:59:08.659128 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.659030    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bc968ab-ab07-49bd-abb7-d1ce58284ea6" containerName="maas-api"
Jul 28 14:59:08.659128 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.659046    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="126006e8-d01d-4bef-a87c-7a697f71e836" containerName="payload-processing"
Jul 28 14:59:08.659128 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.659054    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="126006e8-d01d-4bef-a87c-7a697f71e836" containerName="payload-processing"
Jul 28 14:59:08.659358 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.659154    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="126006e8-d01d-4bef-a87c-7a697f71e836" containerName="payload-processing"
Jul 28 14:59:08.659358 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.659165    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2caca922-1a67-41b4-9a55-06d3a5ea9f26" containerName="payload-pre-processing"
Jul 28 14:59:08.659358 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.659178    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bc968ab-ab07-49bd-abb7-d1ce58284ea6" containerName="maas-api"
Jul 28 14:59:08.662359 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.662340    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.664737 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.664713    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-917bdcc4-openshift-default-dockercfg-rws2k\""
Jul 28 14:59:08.673552 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.673527    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"]
Jul 28 14:59:08.818315 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.818260    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-455fc\" (UniqueName: \"kubernetes.io/projected/1d5282b4-433c-4720-b70c-9977f0209f6f-kube-api-access-455fc\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.818476 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.818339    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-workload-socket\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.818476 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.818389    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-token\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.818580 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.818496    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-envoy\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.818580 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.818528    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-workload-certs\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.818580 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.818547    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-podinfo\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.818732 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.818635    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-credential-socket\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.818732 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.818681    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d5282b4-433c-4720-b70c-9977f0209f6f-istiod-ca-cert\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.818732 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.818721    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-data\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.920150 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.920047    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-token\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.920306 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.920170    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-envoy\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.920306 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.920206    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-workload-certs\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.920306 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.920231    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-podinfo\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.920306 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.920295    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-credential-socket\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.920503 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.920341    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d5282b4-433c-4720-b70c-9977f0209f6f-istiod-ca-cert\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.920503 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.920361    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-data\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.920503 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.920400    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-455fc\" (UniqueName: \"kubernetes.io/projected/1d5282b4-433c-4720-b70c-9977f0209f6f-kube-api-access-455fc\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.921176 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.920707    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-workload-socket\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.921176 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.920772    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-workload-certs\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.921176 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.920781    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-data\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.921176 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.920843    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-credential-socket\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.921176 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.921125    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-workload-socket\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.921431 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.921207    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d5282b4-433c-4720-b70c-9977f0209f6f-istiod-ca-cert\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.922621 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.922596    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-envoy\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.923107 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.923078    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-podinfo\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.927874 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.927853    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-token\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.928205 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.928185    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-455fc\" (UniqueName: \"kubernetes.io/projected/1d5282b4-433c-4720-b70c-9977f0209f6f-kube-api-access-455fc\") pod \"e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") " pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:08.973456 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:08.973416    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:09.113982 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:09.113956    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"]
Jul 28 14:59:09.116560 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:59:09.116535    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d5282b4_433c_4720_b70c_9977f0209f6f.slice/crio-5894d8df019064e077daac216e94c9cb718a9e02e9a846c171cb74331c7c6ea9 WatchSource:0}: Error finding container 5894d8df019064e077daac216e94c9cb718a9e02e9a846c171cb74331c7c6ea9: Status 404 returned error can't find the container with id 5894d8df019064e077daac216e94c9cb718a9e02e9a846c171cb74331c7c6ea9
Jul 28 14:59:09.118948 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:09.118898    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:59:09.119048 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:09.119004    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:59:09.119109 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:09.119050    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:59:09.947320 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:09.947273    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx" event={"ID":"1d5282b4-433c-4720-b70c-9977f0209f6f","Type":"ContainerStarted","Data":"948301e9235981aa7ae46acd5771d7751335f1a118541644848d1bfc7d13dd88"}
Jul 28 14:59:09.947706 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:09.947341    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx" event={"ID":"1d5282b4-433c-4720-b70c-9977f0209f6f","Type":"ContainerStarted","Data":"5894d8df019064e077daac216e94c9cb718a9e02e9a846c171cb74331c7c6ea9"}
Jul 28 14:59:09.969024 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:09.968968    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx" podStartSLOduration=1.968951125 podStartE2EDuration="1.968951125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:59:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:59:09.967181792 +0000 UTC m=+2419.768724086" watchObservedRunningTime="2026-07-28 14:59:09.968951125 +0000 UTC m=+2419.770493339"
Jul 28 14:59:09.973715 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:09.973693    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:10.979510 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:10.979477    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:11.956460 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:11.956428    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:11.957622 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:11.957602    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:12.009266 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.009233    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"]
Jul 28 14:59:12.012915 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.012891    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"
Jul 28 14:59:12.015403 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.015377    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-917bdcc4-serving-cert\""
Jul 28 14:59:12.023815 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.023787    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"]
Jul 28 14:59:12.051284 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.051205    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a3fe41e-6e34-4e3f-8802-181386abc40d-maas-api-tls\") pod \"maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx\" (UID: \"1a3fe41e-6e34-4e3f-8802-181386abc40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"
Jul 28 14:59:12.051284 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.051257    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn6ls\" (UniqueName: \"kubernetes.io/projected/1a3fe41e-6e34-4e3f-8802-181386abc40d-kube-api-access-cn6ls\") pod \"maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx\" (UID: \"1a3fe41e-6e34-4e3f-8802-181386abc40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"
Jul 28 14:59:12.051508 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.051311    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3fe41e-6e34-4e3f-8802-181386abc40d-openshift-service-ca\") pod \"maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx\" (UID: \"1a3fe41e-6e34-4e3f-8802-181386abc40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"
Jul 28 14:59:12.152326 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.152291    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a3fe41e-6e34-4e3f-8802-181386abc40d-maas-api-tls\") pod \"maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx\" (UID: \"1a3fe41e-6e34-4e3f-8802-181386abc40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"
Jul 28 14:59:12.152326 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.152330    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn6ls\" (UniqueName: \"kubernetes.io/projected/1a3fe41e-6e34-4e3f-8802-181386abc40d-kube-api-access-cn6ls\") pod \"maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx\" (UID: \"1a3fe41e-6e34-4e3f-8802-181386abc40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"
Jul 28 14:59:12.152610 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.152366    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3fe41e-6e34-4e3f-8802-181386abc40d-openshift-service-ca\") pod \"maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx\" (UID: \"1a3fe41e-6e34-4e3f-8802-181386abc40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"
Jul 28 14:59:12.153027 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.152994    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3fe41e-6e34-4e3f-8802-181386abc40d-openshift-service-ca\") pod \"maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx\" (UID: \"1a3fe41e-6e34-4e3f-8802-181386abc40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"
Jul 28 14:59:12.154786 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.154765    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a3fe41e-6e34-4e3f-8802-181386abc40d-maas-api-tls\") pod \"maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx\" (UID: \"1a3fe41e-6e34-4e3f-8802-181386abc40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"
Jul 28 14:59:12.163323 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.163302    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn6ls\" (UniqueName: \"kubernetes.io/projected/1a3fe41e-6e34-4e3f-8802-181386abc40d-kube-api-access-cn6ls\") pod \"maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx\" (UID: \"1a3fe41e-6e34-4e3f-8802-181386abc40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"
Jul 28 14:59:12.326532 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.326423    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"
Jul 28 14:59:12.478554 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.478528    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"]
Jul 28 14:59:12.481497 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:59:12.481466    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a3fe41e_6e34_4e3f_8802_181386abc40d.slice/crio-49183eeefd2715a49755a5c5203cad6ad1f62b798c2076eef2a0a39a0bc1c187 WatchSource:0}: Error finding container 49183eeefd2715a49755a5c5203cad6ad1f62b798c2076eef2a0a39a0bc1c187: Status 404 returned error can't find the container with id 49183eeefd2715a49755a5c5203cad6ad1f62b798c2076eef2a0a39a0bc1c187
Jul 28 14:59:12.627820 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.627784    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj"]
Jul 28 14:59:12.632640 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.632616    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj"
Jul 28 14:59:12.634988 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.634948    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-917bdcc4\""
Jul 28 14:59:12.635149 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.635034    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-917bdcc4-dockercfg-jwkpv\""
Jul 28 14:59:12.641848 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.641826    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj"]
Jul 28 14:59:12.656547 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.656513    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c5859f6-2128-4427-aeab-d825e81abf77-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj\" (UID: \"5c5859f6-2128-4427-aeab-d825e81abf77\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj"
Jul 28 14:59:12.656741 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.656624    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7bhc\" (UniqueName: \"kubernetes.io/projected/5c5859f6-2128-4427-aeab-d825e81abf77-kube-api-access-t7bhc\") pod \"payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj\" (UID: \"5c5859f6-2128-4427-aeab-d825e81abf77\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj"
Jul 28 14:59:12.665240 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.665204    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44"]
Jul 28 14:59:12.668476 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.668455    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44"
Jul 28 14:59:12.679153 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.679128    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44"]
Jul 28 14:59:12.757357 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.757314    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b730693d-2d88-48df-add5-150e762f203e-plugins-config-volume\") pod \"payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44\" (UID: \"b730693d-2d88-48df-add5-150e762f203e\") " pod="openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44"
Jul 28 14:59:12.757560 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.757426    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7bhc\" (UniqueName: \"kubernetes.io/projected/5c5859f6-2128-4427-aeab-d825e81abf77-kube-api-access-t7bhc\") pod \"payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj\" (UID: \"5c5859f6-2128-4427-aeab-d825e81abf77\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj"
Jul 28 14:59:12.757560 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.757464    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h4sj\" (UniqueName: \"kubernetes.io/projected/b730693d-2d88-48df-add5-150e762f203e-kube-api-access-8h4sj\") pod \"payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44\" (UID: \"b730693d-2d88-48df-add5-150e762f203e\") " pod="openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44"
Jul 28 14:59:12.757560 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.757526    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c5859f6-2128-4427-aeab-d825e81abf77-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj\" (UID: \"5c5859f6-2128-4427-aeab-d825e81abf77\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj"
Jul 28 14:59:12.758313 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.758288    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c5859f6-2128-4427-aeab-d825e81abf77-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj\" (UID: \"5c5859f6-2128-4427-aeab-d825e81abf77\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj"
Jul 28 14:59:12.765212 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.765192    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7bhc\" (UniqueName: \"kubernetes.io/projected/5c5859f6-2128-4427-aeab-d825e81abf77-kube-api-access-t7bhc\") pod \"payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj\" (UID: \"5c5859f6-2128-4427-aeab-d825e81abf77\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj"
Jul 28 14:59:12.858384 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.858343    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b730693d-2d88-48df-add5-150e762f203e-plugins-config-volume\") pod \"payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44\" (UID: \"b730693d-2d88-48df-add5-150e762f203e\") " pod="openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44"
Jul 28 14:59:12.858567 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.858394    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h4sj\" (UniqueName: \"kubernetes.io/projected/b730693d-2d88-48df-add5-150e762f203e-kube-api-access-8h4sj\") pod \"payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44\" (UID: \"b730693d-2d88-48df-add5-150e762f203e\") " pod="openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44"
Jul 28 14:59:12.859160 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.859078    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b730693d-2d88-48df-add5-150e762f203e-plugins-config-volume\") pod \"payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44\" (UID: \"b730693d-2d88-48df-add5-150e762f203e\") " pod="openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44"
Jul 28 14:59:12.866389 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.866363    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h4sj\" (UniqueName: \"kubernetes.io/projected/b730693d-2d88-48df-add5-150e762f203e-kube-api-access-8h4sj\") pod \"payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44\" (UID: \"b730693d-2d88-48df-add5-150e762f203e\") " pod="openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44"
Jul 28 14:59:12.945006 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.944973    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj"
Jul 28 14:59:12.963361 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.963327    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx" event={"ID":"1a3fe41e-6e34-4e3f-8802-181386abc40d","Type":"ContainerStarted","Data":"49183eeefd2715a49755a5c5203cad6ad1f62b798c2076eef2a0a39a0bc1c187"}
Jul 28 14:59:12.978996 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:12.978967    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44"
Jul 28 14:59:13.100408 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:13.100191    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj"]
Jul 28 14:59:13.103839 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:59:13.103804    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c5859f6_2128_4427_aeab_d825e81abf77.slice/crio-10144bbc4d8549454c12313ebe2efc951d57578d6e8e698e5edf07fa1dd29d46 WatchSource:0}: Error finding container 10144bbc4d8549454c12313ebe2efc951d57578d6e8e698e5edf07fa1dd29d46: Status 404 returned error can't find the container with id 10144bbc4d8549454c12313ebe2efc951d57578d6e8e698e5edf07fa1dd29d46
Jul 28 14:59:13.142361 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:13.142328    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44"]
Jul 28 14:59:13.144896 ip-10-0-130-66 kubenswrapper[2584]: W0728 14:59:13.144867    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb730693d_2d88_48df_add5_150e762f203e.slice/crio-4a5639a4da4a7a4cedf5e49757492d585e6053b98e62b6ac2c8d9ea310d4b8b2 WatchSource:0}: Error finding container 4a5639a4da4a7a4cedf5e49757492d585e6053b98e62b6ac2c8d9ea310d4b8b2: Status 404 returned error can't find the container with id 4a5639a4da4a7a4cedf5e49757492d585e6053b98e62b6ac2c8d9ea310d4b8b2
Jul 28 14:59:13.969471 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:13.969429    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx" event={"ID":"1a3fe41e-6e34-4e3f-8802-181386abc40d","Type":"ContainerStarted","Data":"afe1c259a640ee166b7faaf392ee338f569e01b9bd8a315dde55b4099b18a9f7"}
Jul 28 14:59:13.969697 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:13.969556    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"
Jul 28 14:59:13.971168 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:13.971139    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj" event={"ID":"5c5859f6-2128-4427-aeab-d825e81abf77","Type":"ContainerStarted","Data":"7b26cd808f0964849bbad3f75475cde5d4307788ebd78bfdff685b682feed07c"}
Jul 28 14:59:13.971288 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:13.971195    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj"
Jul 28 14:59:13.971288 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:13.971219    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj" event={"ID":"5c5859f6-2128-4427-aeab-d825e81abf77","Type":"ContainerStarted","Data":"10144bbc4d8549454c12313ebe2efc951d57578d6e8e698e5edf07fa1dd29d46"}
Jul 28 14:59:13.972686 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:13.972663    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44" event={"ID":"b730693d-2d88-48df-add5-150e762f203e","Type":"ContainerStarted","Data":"6087d18818554383b0f029b788f547212d64dc50ee74c67c772cff31fb40557d"}
Jul 28 14:59:13.972795 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:13.972709    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44" event={"ID":"b730693d-2d88-48df-add5-150e762f203e","Type":"ContainerStarted","Data":"4a5639a4da4a7a4cedf5e49757492d585e6053b98e62b6ac2c8d9ea310d4b8b2"}
Jul 28 14:59:13.987458 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:13.987403    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx" podStartSLOduration=2.551615716 podStartE2EDuration="2.987387009s" totalImagesPullingTime="435.771293ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:59:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:59:13.985389522 +0000 UTC m=+2423.786931738" watchObservedRunningTime="2026-07-28 14:59:13.987387009 +0000 UTC m=+2423.788929224"
Jul 28 14:59:14.001805 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:14.001752    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj" podStartSLOduration=2.001735848 podStartE2EDuration="2.001735848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:59:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:59:13.999325107 +0000 UTC m=+2423.800867324" watchObservedRunningTime="2026-07-28 14:59:14.001735848 +0000 UTC m=+2423.803278062"
Jul 28 14:59:14.016848 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:14.016791    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44" podStartSLOduration=2.01677442 podStartE2EDuration="2.01677442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:59:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:59:14.014156975 +0000 UTC m=+2423.815699191" watchObservedRunningTime="2026-07-28 14:59:14.01677442 +0000 UTC m=+2423.818316634"
Jul 28 14:59:14.977484 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:14.977438    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44"
Jul 28 14:59:19.982941 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:19.982902    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"
Jul 28 14:59:24.980193 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:24.980160    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj"
Jul 28 14:59:25.984241 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:25.984217    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44"
Jul 28 14:59:26.035054 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:26.034965    2584 generic.go:356] "Generic (PLEG): container finished" podID="1d5282b4-433c-4720-b70c-9977f0209f6f" containerID="948301e9235981aa7ae46acd5771d7751335f1a118541644848d1bfc7d13dd88" exitCode=0
Jul 28 14:59:26.035054 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:26.035040    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx" event={"ID":"1d5282b4-433c-4720-b70c-9977f0209f6f","Type":"ContainerDied","Data":"948301e9235981aa7ae46acd5771d7751335f1a118541644848d1bfc7d13dd88"}
Jul 28 14:59:26.035531 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:26.035514    2584 scope.go:117] "RemoveContainer" containerID="948301e9235981aa7ae46acd5771d7751335f1a118541644848d1bfc7d13dd88"
Jul 28 14:59:26.036606 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:26.036577    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:59:26.036688 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:26.036644    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:59:26.036688 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:26.036676    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 14:59:26.956782 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:26.956742    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:27.046537 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:27.046501    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx" event={"ID":"1d5282b4-433c-4720-b70c-9977f0209f6f","Type":"ContainerStarted","Data":"5f6fc3c982c2133f67c98acec0c4bc3c44165f5a9c5492ab8cb3480e8da8e581"}
Jul 28 14:59:27.973742 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:27.973705    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:27.974697 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:27.974671    2584 patch_prober.go:28] interesting pod/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 28 14:59:27.974814 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:27.974723    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx" podUID="1d5282b4-433c-4720-b70c-9977f0209f6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 28 14:59:28.974526 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:28.974486    2584 patch_prober.go:28] interesting pod/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 28 14:59:28.974909 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:28.974542    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx" podUID="1d5282b4-433c-4720-b70c-9977f0209f6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 28 14:59:29.974578 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:29.974545    2584 patch_prober.go:28] interesting pod/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 28 14:59:29.974945 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:29.974603    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx" podUID="1d5282b4-433c-4720-b70c-9977f0209f6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 28 14:59:30.974616 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:30.974573    2584 patch_prober.go:28] interesting pod/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 28 14:59:30.975037 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:30.974636    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx" podUID="1d5282b4-433c-4720-b70c-9977f0209f6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 28 14:59:31.977988 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:31.977951    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:31.978402 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:31.978267    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 14:59:31.978840 ip-10-0-130-66 kubenswrapper[2584]: I0728 14:59:31.978822    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 15:00:14.897120 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:14.897065    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"]
Jul 28 15:00:14.897693 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:14.897315    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx" podUID="1a3fe41e-6e34-4e3f-8802-181386abc40d" containerName="maas-api" containerID="cri-o://afe1c259a640ee166b7faaf392ee338f569e01b9bd8a315dde55b4099b18a9f7" gracePeriod=30
Jul 28 15:00:14.958973 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:14.958935    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44"]
Jul 28 15:00:14.959623 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:14.959219    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44" podUID="b730693d-2d88-48df-add5-150e762f203e" containerName="payload-processing" containerID="cri-o://6087d18818554383b0f029b788f547212d64dc50ee74c67c772cff31fb40557d" gracePeriod=30
Jul 28 15:00:14.967242 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:14.967211    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj"]
Jul 28 15:00:14.967472 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:14.967442    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj" podUID="5c5859f6-2128-4427-aeab-d825e81abf77" containerName="payload-pre-processing" containerID="cri-o://7b26cd808f0964849bbad3f75475cde5d4307788ebd78bfdff685b682feed07c" gracePeriod=30
Jul 28 15:00:14.989072 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:14.984916    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx" podUID="1a3fe41e-6e34-4e3f-8802-181386abc40d" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.62:8443/health\": dial tcp 10.134.0.62:8443: connect: connection refused"
Jul 28 15:00:14.989072 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:14.985016    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj" podUID="5c5859f6-2128-4427-aeab-d825e81abf77" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.63:9004: connect: connection refused"
Jul 28 15:00:15.234231 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.234201    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44"
Jul 28 15:00:15.274809 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.274783    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"
Jul 28 15:00:15.277716 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.277696    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj"
Jul 28 15:00:15.289174 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.289142    2584 generic.go:356] "Generic (PLEG): container finished" podID="1a3fe41e-6e34-4e3f-8802-181386abc40d" containerID="afe1c259a640ee166b7faaf392ee338f569e01b9bd8a315dde55b4099b18a9f7" exitCode=0
Jul 28 15:00:15.289303 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.289180    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"
Jul 28 15:00:15.289303 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.289230    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx" event={"ID":"1a3fe41e-6e34-4e3f-8802-181386abc40d","Type":"ContainerDied","Data":"afe1c259a640ee166b7faaf392ee338f569e01b9bd8a315dde55b4099b18a9f7"}
Jul 28 15:00:15.289394 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.289314    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx" event={"ID":"1a3fe41e-6e34-4e3f-8802-181386abc40d","Type":"ContainerDied","Data":"49183eeefd2715a49755a5c5203cad6ad1f62b798c2076eef2a0a39a0bc1c187"}
Jul 28 15:00:15.289394 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.289349    2584 scope.go:117] "RemoveContainer" containerID="afe1c259a640ee166b7faaf392ee338f569e01b9bd8a315dde55b4099b18a9f7"
Jul 28 15:00:15.290384 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.290365    2584 generic.go:356] "Generic (PLEG): container finished" podID="5c5859f6-2128-4427-aeab-d825e81abf77" containerID="7b26cd808f0964849bbad3f75475cde5d4307788ebd78bfdff685b682feed07c" exitCode=0
Jul 28 15:00:15.290491 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.290396    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj" event={"ID":"5c5859f6-2128-4427-aeab-d825e81abf77","Type":"ContainerDied","Data":"7b26cd808f0964849bbad3f75475cde5d4307788ebd78bfdff685b682feed07c"}
Jul 28 15:00:15.290491 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.290414    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj"
Jul 28 15:00:15.290491 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.290444    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj" event={"ID":"5c5859f6-2128-4427-aeab-d825e81abf77","Type":"ContainerDied","Data":"10144bbc4d8549454c12313ebe2efc951d57578d6e8e698e5edf07fa1dd29d46"}
Jul 28 15:00:15.291887 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.291869    2584 generic.go:356] "Generic (PLEG): container finished" podID="b730693d-2d88-48df-add5-150e762f203e" containerID="6087d18818554383b0f029b788f547212d64dc50ee74c67c772cff31fb40557d" exitCode=0
Jul 28 15:00:15.292056 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.291900    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44" event={"ID":"b730693d-2d88-48df-add5-150e762f203e","Type":"ContainerDied","Data":"6087d18818554383b0f029b788f547212d64dc50ee74c67c772cff31fb40557d"}
Jul 28 15:00:15.292205 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.292109    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44" event={"ID":"b730693d-2d88-48df-add5-150e762f203e","Type":"ContainerDied","Data":"4a5639a4da4a7a4cedf5e49757492d585e6053b98e62b6ac2c8d9ea310d4b8b2"}
Jul 28 15:00:15.292205 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.291927    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44"
Jul 28 15:00:15.300849 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.300831    2584 scope.go:117] "RemoveContainer" containerID="afe1c259a640ee166b7faaf392ee338f569e01b9bd8a315dde55b4099b18a9f7"
Jul 28 15:00:15.301135 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:00:15.301115    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afe1c259a640ee166b7faaf392ee338f569e01b9bd8a315dde55b4099b18a9f7\": container with ID starting with afe1c259a640ee166b7faaf392ee338f569e01b9bd8a315dde55b4099b18a9f7 not found: ID does not exist" containerID="afe1c259a640ee166b7faaf392ee338f569e01b9bd8a315dde55b4099b18a9f7"
Jul 28 15:00:15.301187 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.301145    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afe1c259a640ee166b7faaf392ee338f569e01b9bd8a315dde55b4099b18a9f7"} err="failed to get container status \"afe1c259a640ee166b7faaf392ee338f569e01b9bd8a315dde55b4099b18a9f7\": rpc error: code = NotFound desc = could not find container \"afe1c259a640ee166b7faaf392ee338f569e01b9bd8a315dde55b4099b18a9f7\": container with ID starting with afe1c259a640ee166b7faaf392ee338f569e01b9bd8a315dde55b4099b18a9f7 not found: ID does not exist"
Jul 28 15:00:15.301187 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.301162    2584 scope.go:117] "RemoveContainer" containerID="7b26cd808f0964849bbad3f75475cde5d4307788ebd78bfdff685b682feed07c"
Jul 28 15:00:15.309301 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.309276    2584 scope.go:117] "RemoveContainer" containerID="7b26cd808f0964849bbad3f75475cde5d4307788ebd78bfdff685b682feed07c"
Jul 28 15:00:15.309587 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:00:15.309567    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b26cd808f0964849bbad3f75475cde5d4307788ebd78bfdff685b682feed07c\": container with ID starting with 7b26cd808f0964849bbad3f75475cde5d4307788ebd78bfdff685b682feed07c not found: ID does not exist" containerID="7b26cd808f0964849bbad3f75475cde5d4307788ebd78bfdff685b682feed07c"
Jul 28 15:00:15.309657 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.309603    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b26cd808f0964849bbad3f75475cde5d4307788ebd78bfdff685b682feed07c"} err="failed to get container status \"7b26cd808f0964849bbad3f75475cde5d4307788ebd78bfdff685b682feed07c\": rpc error: code = NotFound desc = could not find container \"7b26cd808f0964849bbad3f75475cde5d4307788ebd78bfdff685b682feed07c\": container with ID starting with 7b26cd808f0964849bbad3f75475cde5d4307788ebd78bfdff685b682feed07c not found: ID does not exist"
Jul 28 15:00:15.309657 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.309623    2584 scope.go:117] "RemoveContainer" containerID="6087d18818554383b0f029b788f547212d64dc50ee74c67c772cff31fb40557d"
Jul 28 15:00:15.319900 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.319881    2584 scope.go:117] "RemoveContainer" containerID="6087d18818554383b0f029b788f547212d64dc50ee74c67c772cff31fb40557d"
Jul 28 15:00:15.320262 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:00:15.320239    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6087d18818554383b0f029b788f547212d64dc50ee74c67c772cff31fb40557d\": container with ID starting with 6087d18818554383b0f029b788f547212d64dc50ee74c67c772cff31fb40557d not found: ID does not exist" containerID="6087d18818554383b0f029b788f547212d64dc50ee74c67c772cff31fb40557d"
Jul 28 15:00:15.320358 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.320269    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6087d18818554383b0f029b788f547212d64dc50ee74c67c772cff31fb40557d"} err="failed to get container status \"6087d18818554383b0f029b788f547212d64dc50ee74c67c772cff31fb40557d\": rpc error: code = NotFound desc = could not find container \"6087d18818554383b0f029b788f547212d64dc50ee74c67c772cff31fb40557d\": container with ID starting with 6087d18818554383b0f029b788f547212d64dc50ee74c67c772cff31fb40557d not found: ID does not exist"
Jul 28 15:00:15.342736 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.342712    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c5859f6-2128-4427-aeab-d825e81abf77-plugins-config-volume\") pod \"5c5859f6-2128-4427-aeab-d825e81abf77\" (UID: \"5c5859f6-2128-4427-aeab-d825e81abf77\") "
Jul 28 15:00:15.342852 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.342747    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cn6ls\" (UniqueName: \"kubernetes.io/projected/1a3fe41e-6e34-4e3f-8802-181386abc40d-kube-api-access-cn6ls\") pod \"1a3fe41e-6e34-4e3f-8802-181386abc40d\" (UID: \"1a3fe41e-6e34-4e3f-8802-181386abc40d\") "
Jul 28 15:00:15.342852 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.342766    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7bhc\" (UniqueName: \"kubernetes.io/projected/5c5859f6-2128-4427-aeab-d825e81abf77-kube-api-access-t7bhc\") pod \"5c5859f6-2128-4427-aeab-d825e81abf77\" (UID: \"5c5859f6-2128-4427-aeab-d825e81abf77\") "
Jul 28 15:00:15.342852 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.342838    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b730693d-2d88-48df-add5-150e762f203e-plugins-config-volume\") pod \"b730693d-2d88-48df-add5-150e762f203e\" (UID: \"b730693d-2d88-48df-add5-150e762f203e\") "
Jul 28 15:00:15.342964 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.342861    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8h4sj\" (UniqueName: \"kubernetes.io/projected/b730693d-2d88-48df-add5-150e762f203e-kube-api-access-8h4sj\") pod \"b730693d-2d88-48df-add5-150e762f203e\" (UID: \"b730693d-2d88-48df-add5-150e762f203e\") "
Jul 28 15:00:15.342964 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.342916    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a3fe41e-6e34-4e3f-8802-181386abc40d-maas-api-tls\") pod \"1a3fe41e-6e34-4e3f-8802-181386abc40d\" (UID: \"1a3fe41e-6e34-4e3f-8802-181386abc40d\") "
Jul 28 15:00:15.343065 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.342975    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3fe41e-6e34-4e3f-8802-181386abc40d-openshift-service-ca\") pod \"1a3fe41e-6e34-4e3f-8802-181386abc40d\" (UID: \"1a3fe41e-6e34-4e3f-8802-181386abc40d\") "
Jul 28 15:00:15.343222 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.343190    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c5859f6-2128-4427-aeab-d825e81abf77-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5c5859f6-2128-4427-aeab-d825e81abf77" (UID: "5c5859f6-2128-4427-aeab-d825e81abf77"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:00:15.343397 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.343237    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b730693d-2d88-48df-add5-150e762f203e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b730693d-2d88-48df-add5-150e762f203e" (UID: "b730693d-2d88-48df-add5-150e762f203e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:00:15.343467 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.343419    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a3fe41e-6e34-4e3f-8802-181386abc40d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1a3fe41e-6e34-4e3f-8802-181386abc40d" (UID: "1a3fe41e-6e34-4e3f-8802-181386abc40d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:00:15.343467 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.343455    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c5859f6-2128-4427-aeab-d825e81abf77-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:00:15.344985 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.344958    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a3fe41e-6e34-4e3f-8802-181386abc40d-kube-api-access-cn6ls" (OuterVolumeSpecName: "kube-api-access-cn6ls") pod "1a3fe41e-6e34-4e3f-8802-181386abc40d" (UID: "1a3fe41e-6e34-4e3f-8802-181386abc40d"). InnerVolumeSpecName "kube-api-access-cn6ls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:00:15.344985 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.344965    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b730693d-2d88-48df-add5-150e762f203e-kube-api-access-8h4sj" (OuterVolumeSpecName: "kube-api-access-8h4sj") pod "b730693d-2d88-48df-add5-150e762f203e" (UID: "b730693d-2d88-48df-add5-150e762f203e"). InnerVolumeSpecName "kube-api-access-8h4sj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:00:15.345137 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.345024    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a3fe41e-6e34-4e3f-8802-181386abc40d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1a3fe41e-6e34-4e3f-8802-181386abc40d" (UID: "1a3fe41e-6e34-4e3f-8802-181386abc40d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:00:15.345292 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.345274    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c5859f6-2128-4427-aeab-d825e81abf77-kube-api-access-t7bhc" (OuterVolumeSpecName: "kube-api-access-t7bhc") pod "5c5859f6-2128-4427-aeab-d825e81abf77" (UID: "5c5859f6-2128-4427-aeab-d825e81abf77"). InnerVolumeSpecName "kube-api-access-t7bhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:00:15.444156 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.444050    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b730693d-2d88-48df-add5-150e762f203e-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:00:15.444156 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.444076    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8h4sj\" (UniqueName: \"kubernetes.io/projected/b730693d-2d88-48df-add5-150e762f203e-kube-api-access-8h4sj\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:00:15.444156 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.444086    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a3fe41e-6e34-4e3f-8802-181386abc40d-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:00:15.444156 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.444122    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3fe41e-6e34-4e3f-8802-181386abc40d-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:00:15.444156 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.444131    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn6ls\" (UniqueName: \"kubernetes.io/projected/1a3fe41e-6e34-4e3f-8802-181386abc40d-kube-api-access-cn6ls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:00:15.444156 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.444139    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7bhc\" (UniqueName: \"kubernetes.io/projected/5c5859f6-2128-4427-aeab-d825e81abf77-kube-api-access-t7bhc\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:00:15.613136 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.613108    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"]
Jul 28 15:00:15.616435 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.616414    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-917bdcc4-64d7b6b4b4-8stxx"]
Jul 28 15:00:15.626175 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.626149    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj"]
Jul 28 15:00:15.629462 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.629441    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-917bdcc4-6b564c46d4-4h6hj"]
Jul 28 15:00:15.639470 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.639448    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44"]
Jul 28 15:00:15.643139 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:15.643120    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-917bdcc4-79744b7c9d-cpq44"]
Jul 28 15:00:16.795864 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:16.795831    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a3fe41e-6e34-4e3f-8802-181386abc40d" path="/var/lib/kubelet/pods/1a3fe41e-6e34-4e3f-8802-181386abc40d/volumes"
Jul 28 15:00:16.796356 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:16.796204    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c5859f6-2128-4427-aeab-d825e81abf77" path="/var/lib/kubelet/pods/5c5859f6-2128-4427-aeab-d825e81abf77/volumes"
Jul 28 15:00:16.796568 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:00:16.796553    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b730693d-2d88-48df-add5-150e762f203e" path="/var/lib/kubelet/pods/b730693d-2d88-48df-add5-150e762f203e/volumes"
Jul 28 15:01:33.629361 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:33.629322    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"]
Jul 28 15:01:33.630342 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:33.630300    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx" podUID="1d5282b4-433c-4720-b70c-9977f0209f6f" containerName="istio-proxy" containerID="cri-o://5f6fc3c982c2133f67c98acec0c4bc3c44165f5a9c5492ab8cb3480e8da8e581" gracePeriod=30
Jul 28 15:01:38.889955 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:38.889924    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 15:01:39.024154 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.024072    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-data\") pod \"1d5282b4-433c-4720-b70c-9977f0209f6f\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") "
Jul 28 15:01:39.024154 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.024133    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-workload-certs\") pod \"1d5282b4-433c-4720-b70c-9977f0209f6f\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") "
Jul 28 15:01:39.024404 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.024172    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-token\") pod \"1d5282b4-433c-4720-b70c-9977f0209f6f\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") "
Jul 28 15:01:39.024404 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.024206    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-podinfo\") pod \"1d5282b4-433c-4720-b70c-9977f0209f6f\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") "
Jul 28 15:01:39.024404 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.024230    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-credential-socket\") pod \"1d5282b4-433c-4720-b70c-9977f0209f6f\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") "
Jul 28 15:01:39.024404 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.024256    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-workload-socket\") pod \"1d5282b4-433c-4720-b70c-9977f0209f6f\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") "
Jul 28 15:01:39.024404 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.024301    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-envoy\") pod \"1d5282b4-433c-4720-b70c-9977f0209f6f\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") "
Jul 28 15:01:39.024404 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.024328    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d5282b4-433c-4720-b70c-9977f0209f6f-istiod-ca-cert\") pod \"1d5282b4-433c-4720-b70c-9977f0209f6f\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") "
Jul 28 15:01:39.024404 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.024359    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-455fc\" (UniqueName: \"kubernetes.io/projected/1d5282b4-433c-4720-b70c-9977f0209f6f-kube-api-access-455fc\") pod \"1d5282b4-433c-4720-b70c-9977f0209f6f\" (UID: \"1d5282b4-433c-4720-b70c-9977f0209f6f\") "
Jul 28 15:01:39.024404 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.024385    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1d5282b4-433c-4720-b70c-9977f0209f6f" (UID: "1d5282b4-433c-4720-b70c-9977f0209f6f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:01:39.024846 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.024532    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1d5282b4-433c-4720-b70c-9977f0209f6f" (UID: "1d5282b4-433c-4720-b70c-9977f0209f6f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:01:39.024846 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.024819    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-workload-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:01:39.024846 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.024828    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d5282b4-433c-4720-b70c-9977f0209f6f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1d5282b4-433c-4720-b70c-9977f0209f6f" (UID: "1d5282b4-433c-4720-b70c-9977f0209f6f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:01:39.024846 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.024840    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-credential-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:01:39.025019 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.024905    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1d5282b4-433c-4720-b70c-9977f0209f6f" (UID: "1d5282b4-433c-4720-b70c-9977f0209f6f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:01:39.025019 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.024907    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-data" (OuterVolumeSpecName: "istio-data") pod "1d5282b4-433c-4720-b70c-9977f0209f6f" (UID: "1d5282b4-433c-4720-b70c-9977f0209f6f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:01:39.026496 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.026470    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1d5282b4-433c-4720-b70c-9977f0209f6f" (UID: "1d5282b4-433c-4720-b70c-9977f0209f6f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 15:01:39.026630 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.026514    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-token" (OuterVolumeSpecName: "istio-token") pod "1d5282b4-433c-4720-b70c-9977f0209f6f" (UID: "1d5282b4-433c-4720-b70c-9977f0209f6f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:01:39.026856 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.026839    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1d5282b4-433c-4720-b70c-9977f0209f6f" (UID: "1d5282b4-433c-4720-b70c-9977f0209f6f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:01:39.026895 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.026852    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d5282b4-433c-4720-b70c-9977f0209f6f-kube-api-access-455fc" (OuterVolumeSpecName: "kube-api-access-455fc") pod "1d5282b4-433c-4720-b70c-9977f0209f6f" (UID: "1d5282b4-433c-4720-b70c-9977f0209f6f"). InnerVolumeSpecName "kube-api-access-455fc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:01:39.125432 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.125401    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-data\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:01:39.125432 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.125429    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-token\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:01:39.125432 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.125438    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-podinfo\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:01:39.125432 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.125447    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-workload-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:01:39.125711 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.125455    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d5282b4-433c-4720-b70c-9977f0209f6f-istio-envoy\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:01:39.125711 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.125463    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d5282b4-433c-4720-b70c-9977f0209f6f-istiod-ca-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:01:39.125711 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.125471    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-455fc\" (UniqueName: \"kubernetes.io/projected/1d5282b4-433c-4720-b70c-9977f0209f6f-kube-api-access-455fc\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:01:39.704502 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.704464    2584 generic.go:356] "Generic (PLEG): container finished" podID="1d5282b4-433c-4720-b70c-9977f0209f6f" containerID="5f6fc3c982c2133f67c98acec0c4bc3c44165f5a9c5492ab8cb3480e8da8e581" exitCode=0
Jul 28 15:01:39.704679 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.704514    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"
Jul 28 15:01:39.704679 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.704551    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx" event={"ID":"1d5282b4-433c-4720-b70c-9977f0209f6f","Type":"ContainerDied","Data":"5f6fc3c982c2133f67c98acec0c4bc3c44165f5a9c5492ab8cb3480e8da8e581"}
Jul 28 15:01:39.704679 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.704609    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx" event={"ID":"1d5282b4-433c-4720-b70c-9977f0209f6f","Type":"ContainerDied","Data":"5894d8df019064e077daac216e94c9cb718a9e02e9a846c171cb74331c7c6ea9"}
Jul 28 15:01:39.704679 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.704633    2584 scope.go:117] "RemoveContainer" containerID="5f6fc3c982c2133f67c98acec0c4bc3c44165f5a9c5492ab8cb3480e8da8e581"
Jul 28 15:01:39.713631 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.713609    2584 scope.go:117] "RemoveContainer" containerID="948301e9235981aa7ae46acd5771d7751335f1a118541644848d1bfc7d13dd88"
Jul 28 15:01:39.721439 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.721413    2584 scope.go:117] "RemoveContainer" containerID="5f6fc3c982c2133f67c98acec0c4bc3c44165f5a9c5492ab8cb3480e8da8e581"
Jul 28 15:01:39.721716 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:01:39.721662    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f6fc3c982c2133f67c98acec0c4bc3c44165f5a9c5492ab8cb3480e8da8e581\": container with ID starting with 5f6fc3c982c2133f67c98acec0c4bc3c44165f5a9c5492ab8cb3480e8da8e581 not found: ID does not exist" containerID="5f6fc3c982c2133f67c98acec0c4bc3c44165f5a9c5492ab8cb3480e8da8e581"
Jul 28 15:01:39.721774 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.721725    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f6fc3c982c2133f67c98acec0c4bc3c44165f5a9c5492ab8cb3480e8da8e581"} err="failed to get container status \"5f6fc3c982c2133f67c98acec0c4bc3c44165f5a9c5492ab8cb3480e8da8e581\": rpc error: code = NotFound desc = could not find container \"5f6fc3c982c2133f67c98acec0c4bc3c44165f5a9c5492ab8cb3480e8da8e581\": container with ID starting with 5f6fc3c982c2133f67c98acec0c4bc3c44165f5a9c5492ab8cb3480e8da8e581 not found: ID does not exist"
Jul 28 15:01:39.721774 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.721743    2584 scope.go:117] "RemoveContainer" containerID="948301e9235981aa7ae46acd5771d7751335f1a118541644848d1bfc7d13dd88"
Jul 28 15:01:39.721978 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:01:39.721957    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"948301e9235981aa7ae46acd5771d7751335f1a118541644848d1bfc7d13dd88\": container with ID starting with 948301e9235981aa7ae46acd5771d7751335f1a118541644848d1bfc7d13dd88 not found: ID does not exist" containerID="948301e9235981aa7ae46acd5771d7751335f1a118541644848d1bfc7d13dd88"
Jul 28 15:01:39.722024 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.721984    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"948301e9235981aa7ae46acd5771d7751335f1a118541644848d1bfc7d13dd88"} err="failed to get container status \"948301e9235981aa7ae46acd5771d7751335f1a118541644848d1bfc7d13dd88\": rpc error: code = NotFound desc = could not find container \"948301e9235981aa7ae46acd5771d7751335f1a118541644848d1bfc7d13dd88\": container with ID starting with 948301e9235981aa7ae46acd5771d7751335f1a118541644848d1bfc7d13dd88 not found: ID does not exist"
Jul 28 15:01:39.760862 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.760827    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"]
Jul 28 15:01:39.784109 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:39.784068    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-917bdcc4-openshift-default-647d69664-vh4cx"]
Jul 28 15:01:40.795950 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:01:40.795916    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d5282b4-433c-4720-b70c-9977f0209f6f" path="/var/lib/kubelet/pods/1d5282b4-433c-4720-b70c-9977f0209f6f/volumes"
Jul 28 15:07:32.562078 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.562044    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"]
Jul 28 15:07:32.562555 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.562417    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a3fe41e-6e34-4e3f-8802-181386abc40d" containerName="maas-api"
Jul 28 15:07:32.562555 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.562430    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a3fe41e-6e34-4e3f-8802-181386abc40d" containerName="maas-api"
Jul 28 15:07:32.562555 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.562443    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b730693d-2d88-48df-add5-150e762f203e" containerName="payload-processing"
Jul 28 15:07:32.562555 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.562450    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b730693d-2d88-48df-add5-150e762f203e" containerName="payload-processing"
Jul 28 15:07:32.562555 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.562464    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d5282b4-433c-4720-b70c-9977f0209f6f" containerName="istio-proxy"
Jul 28 15:07:32.562555 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.562469    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d5282b4-433c-4720-b70c-9977f0209f6f" containerName="istio-proxy"
Jul 28 15:07:32.562555 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.562478    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d5282b4-433c-4720-b70c-9977f0209f6f" containerName="istio-proxy"
Jul 28 15:07:32.562555 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.562483    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d5282b4-433c-4720-b70c-9977f0209f6f" containerName="istio-proxy"
Jul 28 15:07:32.562555 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.562492    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c5859f6-2128-4427-aeab-d825e81abf77" containerName="payload-pre-processing"
Jul 28 15:07:32.562555 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.562497    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c5859f6-2128-4427-aeab-d825e81abf77" containerName="payload-pre-processing"
Jul 28 15:07:32.562555 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.562547    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b730693d-2d88-48df-add5-150e762f203e" containerName="payload-processing"
Jul 28 15:07:32.562555 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.562555    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a3fe41e-6e34-4e3f-8802-181386abc40d" containerName="maas-api"
Jul 28 15:07:32.562907 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.562563    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c5859f6-2128-4427-aeab-d825e81abf77" containerName="payload-pre-processing"
Jul 28 15:07:32.562907 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.562570    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d5282b4-433c-4720-b70c-9977f0209f6f" containerName="istio-proxy"
Jul 28 15:07:32.562907 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.562575    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d5282b4-433c-4720-b70c-9977f0209f6f" containerName="istio-proxy"
Jul 28 15:07:32.565525 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.565505    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"
Jul 28 15:07:32.567903 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.567874    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-06cd69-serving-cert\""
Jul 28 15:07:32.576774 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.576747    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"]
Jul 28 15:07:32.687459 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.687429    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv5d4\" (UniqueName: \"kubernetes.io/projected/aeb069e6-12a6-4c8b-a989-adef9d7deabd-kube-api-access-zv5d4\") pod \"maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk\" (UID: \"aeb069e6-12a6-4c8b-a989-adef9d7deabd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"
Jul 28 15:07:32.687643 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.687513    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aeb069e6-12a6-4c8b-a989-adef9d7deabd-maas-api-tls\") pod \"maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk\" (UID: \"aeb069e6-12a6-4c8b-a989-adef9d7deabd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"
Jul 28 15:07:32.687643 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.687550    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeb069e6-12a6-4c8b-a989-adef9d7deabd-openshift-service-ca\") pod \"maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk\" (UID: \"aeb069e6-12a6-4c8b-a989-adef9d7deabd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"
Jul 28 15:07:32.788678 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.788640    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv5d4\" (UniqueName: \"kubernetes.io/projected/aeb069e6-12a6-4c8b-a989-adef9d7deabd-kube-api-access-zv5d4\") pod \"maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk\" (UID: \"aeb069e6-12a6-4c8b-a989-adef9d7deabd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"
Jul 28 15:07:32.788863 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.788686    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aeb069e6-12a6-4c8b-a989-adef9d7deabd-maas-api-tls\") pod \"maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk\" (UID: \"aeb069e6-12a6-4c8b-a989-adef9d7deabd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"
Jul 28 15:07:32.788863 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.788708    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeb069e6-12a6-4c8b-a989-adef9d7deabd-openshift-service-ca\") pod \"maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk\" (UID: \"aeb069e6-12a6-4c8b-a989-adef9d7deabd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"
Jul 28 15:07:32.789330 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.789306    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeb069e6-12a6-4c8b-a989-adef9d7deabd-openshift-service-ca\") pod \"maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk\" (UID: \"aeb069e6-12a6-4c8b-a989-adef9d7deabd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"
Jul 28 15:07:32.791397 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.791373    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aeb069e6-12a6-4c8b-a989-adef9d7deabd-maas-api-tls\") pod \"maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk\" (UID: \"aeb069e6-12a6-4c8b-a989-adef9d7deabd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"
Jul 28 15:07:32.796765 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.796743    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv5d4\" (UniqueName: \"kubernetes.io/projected/aeb069e6-12a6-4c8b-a989-adef9d7deabd-kube-api-access-zv5d4\") pod \"maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk\" (UID: \"aeb069e6-12a6-4c8b-a989-adef9d7deabd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"
Jul 28 15:07:32.878466 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:32.878438    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"
Jul 28 15:07:33.013078 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.013042    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"]
Jul 28 15:07:33.015945 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:07:33.015911    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeb069e6_12a6_4c8b_a989_adef9d7deabd.slice/crio-2245ca820286207b1d2ced6e348776cc9088db29d4b7327de87b951dd40753f5 WatchSource:0}: Error finding container 2245ca820286207b1d2ced6e348776cc9088db29d4b7327de87b951dd40753f5: Status 404 returned error can't find the container with id 2245ca820286207b1d2ced6e348776cc9088db29d4b7327de87b951dd40753f5
Jul 28 15:07:33.017640 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.017620    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 15:07:33.246903 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.246808    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn"]
Jul 28 15:07:33.251497 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.251467    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn"
Jul 28 15:07:33.253989 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.253967    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-06cd69\""
Jul 28 15:07:33.262138 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.262108    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn"]
Jul 28 15:07:33.288321 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.288288    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n"]
Jul 28 15:07:33.291613 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.291596    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n"
Jul 28 15:07:33.295017 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.294995    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-06cd69-dockercfg-5nn56\""
Jul 28 15:07:33.303307 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.303279    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n"]
Jul 28 15:07:33.314650 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.314625    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk" event={"ID":"aeb069e6-12a6-4c8b-a989-adef9d7deabd","Type":"ContainerStarted","Data":"2245ca820286207b1d2ced6e348776cc9088db29d4b7327de87b951dd40753f5"}
Jul 28 15:07:33.395412 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.395378    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-776wh\" (UniqueName: \"kubernetes.io/projected/74884c15-557b-4ae0-b667-725b998e61d9-kube-api-access-776wh\") pod \"payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n\" (UID: \"74884c15-557b-4ae0-b667-725b998e61d9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n"
Jul 28 15:07:33.395594 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.395426    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kkwg\" (UniqueName: \"kubernetes.io/projected/a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f-kube-api-access-8kkwg\") pod \"payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn\" (UID: \"a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn"
Jul 28 15:07:33.395594 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.395458    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74884c15-557b-4ae0-b667-725b998e61d9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n\" (UID: \"74884c15-557b-4ae0-b667-725b998e61d9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n"
Jul 28 15:07:33.395594 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.395530    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn\" (UID: \"a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn"
Jul 28 15:07:33.496480 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.496441    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-776wh\" (UniqueName: \"kubernetes.io/projected/74884c15-557b-4ae0-b667-725b998e61d9-kube-api-access-776wh\") pod \"payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n\" (UID: \"74884c15-557b-4ae0-b667-725b998e61d9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n"
Jul 28 15:07:33.496691 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.496491    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kkwg\" (UniqueName: \"kubernetes.io/projected/a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f-kube-api-access-8kkwg\") pod \"payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn\" (UID: \"a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn"
Jul 28 15:07:33.496691 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.496516    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74884c15-557b-4ae0-b667-725b998e61d9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n\" (UID: \"74884c15-557b-4ae0-b667-725b998e61d9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n"
Jul 28 15:07:33.496691 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.496630    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn\" (UID: \"a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn"
Jul 28 15:07:33.497326 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.497262    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74884c15-557b-4ae0-b667-725b998e61d9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n\" (UID: \"74884c15-557b-4ae0-b667-725b998e61d9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n"
Jul 28 15:07:33.497460 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.497341    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn\" (UID: \"a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn"
Jul 28 15:07:33.505201 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.505180    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kkwg\" (UniqueName: \"kubernetes.io/projected/a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f-kube-api-access-8kkwg\") pod \"payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn\" (UID: \"a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn"
Jul 28 15:07:33.505301 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.505223    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-776wh\" (UniqueName: \"kubernetes.io/projected/74884c15-557b-4ae0-b667-725b998e61d9-kube-api-access-776wh\") pod \"payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n\" (UID: \"74884c15-557b-4ae0-b667-725b998e61d9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n"
Jul 28 15:07:33.562337 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.562312    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn"
Jul 28 15:07:33.601643 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.601613    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n"
Jul 28 15:07:33.722952 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.721624    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn"]
Jul 28 15:07:33.724505 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:07:33.724448    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda739e9e9_acb6_4aa6_bb00_bc8bf6a6e56f.slice/crio-d1d49d7dd57edfa8373e636f17992560b62cd3bb40f8b7312486596022a7ad8c WatchSource:0}: Error finding container d1d49d7dd57edfa8373e636f17992560b62cd3bb40f8b7312486596022a7ad8c: Status 404 returned error can't find the container with id d1d49d7dd57edfa8373e636f17992560b62cd3bb40f8b7312486596022a7ad8c
Jul 28 15:07:33.763895 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:33.763867    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n"]
Jul 28 15:07:33.765753 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:07:33.765729    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74884c15_557b_4ae0_b667_725b998e61d9.slice/crio-e51c79609fc15cea27867db260d162e544cb57f0cc0cc55334b674d4e4ee3c2f WatchSource:0}: Error finding container e51c79609fc15cea27867db260d162e544cb57f0cc0cc55334b674d4e4ee3c2f: Status 404 returned error can't find the container with id e51c79609fc15cea27867db260d162e544cb57f0cc0cc55334b674d4e4ee3c2f
Jul 28 15:07:34.320404 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:34.320368    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n" event={"ID":"74884c15-557b-4ae0-b667-725b998e61d9","Type":"ContainerStarted","Data":"ef3b50e424341eb11abf712e2d31f13cc4f9835cfb46ee8143f3c116d499d0df"}
Jul 28 15:07:34.320593 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:34.320432    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n" event={"ID":"74884c15-557b-4ae0-b667-725b998e61d9","Type":"ContainerStarted","Data":"e51c79609fc15cea27867db260d162e544cb57f0cc0cc55334b674d4e4ee3c2f"}
Jul 28 15:07:34.320593 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:34.320549    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n"
Jul 28 15:07:34.321969 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:34.321937    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn" event={"ID":"a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f","Type":"ContainerStarted","Data":"317bb58ffb8c6fb85cc55e50d68a3dc01bd66aa0560ac42cd770deef528ef538"}
Jul 28 15:07:34.322139 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:34.321990    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn" event={"ID":"a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f","Type":"ContainerStarted","Data":"d1d49d7dd57edfa8373e636f17992560b62cd3bb40f8b7312486596022a7ad8c"}
Jul 28 15:07:34.322139 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:34.322017    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn"
Jul 28 15:07:34.323249 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:34.323224    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk" event={"ID":"aeb069e6-12a6-4c8b-a989-adef9d7deabd","Type":"ContainerStarted","Data":"534b86911e3f2678506a55c6dabf5597a802501edc0ebd9acefbe1af0446a62a"}
Jul 28 15:07:34.323380 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:34.323362    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"
Jul 28 15:07:34.339857 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:34.339809    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n" podStartSLOduration=1.339790549 podStartE2EDuration="1.339790549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:07:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:07:34.337582175 +0000 UTC m=+2924.139124391" watchObservedRunningTime="2026-07-28 15:07:34.339790549 +0000 UTC m=+2924.141332759"
Jul 28 15:07:34.356306 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:34.356263    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn" podStartSLOduration=1.3562519929999999 podStartE2EDuration="1.356251993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:07:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:07:34.353437949 +0000 UTC m=+2924.154980162" watchObservedRunningTime="2026-07-28 15:07:34.356251993 +0000 UTC m=+2924.157794209"
Jul 28 15:07:34.376391 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:34.376348    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk" podStartSLOduration=1.8661550230000001 podStartE2EDuration="2.376335404s" totalImagesPullingTime="510.180381ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:07:34.375830165 +0000 UTC m=+2924.177372379" watchObservedRunningTime="2026-07-28 15:07:34.376335404 +0000 UTC m=+2924.177877621"
Jul 28 15:07:40.333377 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:40.333350    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"
Jul 28 15:07:45.329435 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:45.329405    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn"
Jul 28 15:07:45.329877 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:45.329536    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n"
Jul 28 15:07:58.140834 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.140801    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"]
Jul 28 15:07:58.146057 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.146031    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.149212 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.149191    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-f4853e-openshift-default-dockercfg-pjstj\""
Jul 28 15:07:58.159655 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.159626    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"]
Jul 28 15:07:58.231940 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.231903    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-envoy\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.231940 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.231954    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-credential-socket\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.232237 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.231979    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-token\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.232237 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.232088    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-workload-certs\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.232237 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.232157    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-data\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.232237 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.232216    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-podinfo\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.232462 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.232271    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-workload-socket\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.232462 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.232310    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istiod-ca-cert\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.232462 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.232334    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5qm4\" (UniqueName: \"kubernetes.io/projected/1f12e0d7-1a88-426c-b008-196ba2bd65c2-kube-api-access-n5qm4\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.333672 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.333637    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-workload-socket\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.333853 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.333678    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istiod-ca-cert\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.333853 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.333705    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5qm4\" (UniqueName: \"kubernetes.io/projected/1f12e0d7-1a88-426c-b008-196ba2bd65c2-kube-api-access-n5qm4\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.333853 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.333742    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-envoy\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.333853 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.333793    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-credential-socket\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.333853 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.333826    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-token\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.334153 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.333857    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-workload-certs\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.334153 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.333888    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-data\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.334153 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.333994    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-podinfo\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.334153 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.334130    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-workload-socket\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.334390 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.334183    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-credential-socket\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.334442 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.334403    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-workload-certs\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.334442 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.334424    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-data\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.334554 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.334533    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istiod-ca-cert\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.336156 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.336131    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-envoy\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.336353 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.336338    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-podinfo\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.342690 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.342669    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5qm4\" (UniqueName: \"kubernetes.io/projected/1f12e0d7-1a88-426c-b008-196ba2bd65c2-kube-api-access-n5qm4\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.342787 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.342739    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-token\") pod \"e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") " pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.461301 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.461213    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:58.600742 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.600715    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"]
Jul 28 15:07:58.602942 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:07:58.602907    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f12e0d7_1a88_426c_b008_196ba2bd65c2.slice/crio-acba3f0b9195a5fc49132c8e988d26ed50a6caf6cf9536f7a870399c68389a1d WatchSource:0}: Error finding container acba3f0b9195a5fc49132c8e988d26ed50a6caf6cf9536f7a870399c68389a1d: Status 404 returned error can't find the container with id acba3f0b9195a5fc49132c8e988d26ed50a6caf6cf9536f7a870399c68389a1d
Jul 28 15:07:58.605147 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.605116    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 15:07:58.605227 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.605186    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 15:07:58.605227 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:58.605216    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 15:07:59.443007 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:59.442967    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt" event={"ID":"1f12e0d7-1a88-426c-b008-196ba2bd65c2","Type":"ContainerStarted","Data":"6350e61297601e9ae95d71e0c9670e0c8834c311754c63db3579c17b757002cf"}
Jul 28 15:07:59.443403 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:59.443030    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt" event={"ID":"1f12e0d7-1a88-426c-b008-196ba2bd65c2","Type":"ContainerStarted","Data":"acba3f0b9195a5fc49132c8e988d26ed50a6caf6cf9536f7a870399c68389a1d"}
Jul 28 15:07:59.461531 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:59.461502    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:07:59.464653 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:59.464600    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt" podStartSLOduration=1.464582117 podStartE2EDuration="1.464582117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:07:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:07:59.46185728 +0000 UTC m=+2949.263399494" watchObservedRunningTime="2026-07-28 15:07:59.464582117 +0000 UTC m=+2949.266124334"
Jul 28 15:07:59.466721 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:07:59.466696    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:08:00.447932 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:00.447894    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:08:00.448779 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:00.448756    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:08:01.654541 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:01.654506    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"]
Jul 28 15:08:01.657943 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:01.657923    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"
Jul 28 15:08:01.660811 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:01.660781    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-f4853e-serving-cert\""
Jul 28 15:08:01.676290 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:01.676260    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"]
Jul 28 15:08:01.766406 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:01.766364    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef57fd68-f7c2-484a-9d80-3dd3a068b7c9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn\" (UID: \"ef57fd68-f7c2-484a-9d80-3dd3a068b7c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"
Jul 28 15:08:01.766593 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:01.766445    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hxlc\" (UniqueName: \"kubernetes.io/projected/ef57fd68-f7c2-484a-9d80-3dd3a068b7c9-kube-api-access-6hxlc\") pod \"maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn\" (UID: \"ef57fd68-f7c2-484a-9d80-3dd3a068b7c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"
Jul 28 15:08:01.766593 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:01.766502    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef57fd68-f7c2-484a-9d80-3dd3a068b7c9-maas-api-tls\") pod \"maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn\" (UID: \"ef57fd68-f7c2-484a-9d80-3dd3a068b7c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"
Jul 28 15:08:01.867259 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:01.867216    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef57fd68-f7c2-484a-9d80-3dd3a068b7c9-maas-api-tls\") pod \"maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn\" (UID: \"ef57fd68-f7c2-484a-9d80-3dd3a068b7c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"
Jul 28 15:08:01.867476 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:01.867272    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef57fd68-f7c2-484a-9d80-3dd3a068b7c9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn\" (UID: \"ef57fd68-f7c2-484a-9d80-3dd3a068b7c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"
Jul 28 15:08:01.867476 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:01.867342    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hxlc\" (UniqueName: \"kubernetes.io/projected/ef57fd68-f7c2-484a-9d80-3dd3a068b7c9-kube-api-access-6hxlc\") pod \"maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn\" (UID: \"ef57fd68-f7c2-484a-9d80-3dd3a068b7c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"
Jul 28 15:08:01.868043 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:01.868018    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef57fd68-f7c2-484a-9d80-3dd3a068b7c9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn\" (UID: \"ef57fd68-f7c2-484a-9d80-3dd3a068b7c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"
Jul 28 15:08:01.869648 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:01.869628    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef57fd68-f7c2-484a-9d80-3dd3a068b7c9-maas-api-tls\") pod \"maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn\" (UID: \"ef57fd68-f7c2-484a-9d80-3dd3a068b7c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"
Jul 28 15:08:01.876413 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:01.876355    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hxlc\" (UniqueName: \"kubernetes.io/projected/ef57fd68-f7c2-484a-9d80-3dd3a068b7c9-kube-api-access-6hxlc\") pod \"maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn\" (UID: \"ef57fd68-f7c2-484a-9d80-3dd3a068b7c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"
Jul 28 15:08:01.968469 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:01.968384    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"
Jul 28 15:08:02.115347 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.115186    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"]
Jul 28 15:08:02.118123 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:08:02.118084    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef57fd68_f7c2_484a_9d80_3dd3a068b7c9.slice/crio-781c8ea2f2dd560cec1a1c6829728516bf0b87ed27b1b9bcbf295afe3c35b8cd WatchSource:0}: Error finding container 781c8ea2f2dd560cec1a1c6829728516bf0b87ed27b1b9bcbf295afe3c35b8cd: Status 404 returned error can't find the container with id 781c8ea2f2dd560cec1a1c6829728516bf0b87ed27b1b9bcbf295afe3c35b8cd
Jul 28 15:08:02.289346 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.289270    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766"]
Jul 28 15:08:02.293981 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.293961    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766"
Jul 28 15:08:02.296749 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.296727    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-f4853e-dockercfg-x4rtd\""
Jul 28 15:08:02.296859 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.296729    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-f4853e\""
Jul 28 15:08:02.303878 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.303838    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766"]
Jul 28 15:08:02.307298 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.307273    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c"]
Jul 28 15:08:02.310809 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.310792    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c"
Jul 28 15:08:02.324562 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.324537    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c"]
Jul 28 15:08:02.371499 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.371462    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgxmg\" (UniqueName: \"kubernetes.io/projected/e286c623-e2d5-4206-ad16-3053b95d7ac4-kube-api-access-tgxmg\") pod \"payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766\" (UID: \"e286c623-e2d5-4206-ad16-3053b95d7ac4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766"
Jul 28 15:08:02.371680 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.371513    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e286c623-e2d5-4206-ad16-3053b95d7ac4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766\" (UID: \"e286c623-e2d5-4206-ad16-3053b95d7ac4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766"
Jul 28 15:08:02.459310 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.459274    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn" event={"ID":"ef57fd68-f7c2-484a-9d80-3dd3a068b7c9","Type":"ContainerStarted","Data":"781c8ea2f2dd560cec1a1c6829728516bf0b87ed27b1b9bcbf295afe3c35b8cd"}
Jul 28 15:08:02.472987 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.472956    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e286c623-e2d5-4206-ad16-3053b95d7ac4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766\" (UID: \"e286c623-e2d5-4206-ad16-3053b95d7ac4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766"
Jul 28 15:08:02.473178 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.473005    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfj5g\" (UniqueName: \"kubernetes.io/projected/f1a597d7-5f58-40a0-82bb-8bfd563fa48f-kube-api-access-vfj5g\") pod \"payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c\" (UID: \"f1a597d7-5f58-40a0-82bb-8bfd563fa48f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c"
Jul 28 15:08:02.473178 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.473078    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1a597d7-5f58-40a0-82bb-8bfd563fa48f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c\" (UID: \"f1a597d7-5f58-40a0-82bb-8bfd563fa48f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c"
Jul 28 15:08:02.473300 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.473223    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgxmg\" (UniqueName: \"kubernetes.io/projected/e286c623-e2d5-4206-ad16-3053b95d7ac4-kube-api-access-tgxmg\") pod \"payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766\" (UID: \"e286c623-e2d5-4206-ad16-3053b95d7ac4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766"
Jul 28 15:08:02.473708 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.473683    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e286c623-e2d5-4206-ad16-3053b95d7ac4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766\" (UID: \"e286c623-e2d5-4206-ad16-3053b95d7ac4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766"
Jul 28 15:08:02.485816 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.485787    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgxmg\" (UniqueName: \"kubernetes.io/projected/e286c623-e2d5-4206-ad16-3053b95d7ac4-kube-api-access-tgxmg\") pod \"payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766\" (UID: \"e286c623-e2d5-4206-ad16-3053b95d7ac4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766"
Jul 28 15:08:02.574470 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.574371    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfj5g\" (UniqueName: \"kubernetes.io/projected/f1a597d7-5f58-40a0-82bb-8bfd563fa48f-kube-api-access-vfj5g\") pod \"payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c\" (UID: \"f1a597d7-5f58-40a0-82bb-8bfd563fa48f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c"
Jul 28 15:08:02.574705 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.574470    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1a597d7-5f58-40a0-82bb-8bfd563fa48f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c\" (UID: \"f1a597d7-5f58-40a0-82bb-8bfd563fa48f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c"
Jul 28 15:08:02.575166 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.575146    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1a597d7-5f58-40a0-82bb-8bfd563fa48f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c\" (UID: \"f1a597d7-5f58-40a0-82bb-8bfd563fa48f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c"
Jul 28 15:08:02.584733 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.584710    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfj5g\" (UniqueName: \"kubernetes.io/projected/f1a597d7-5f58-40a0-82bb-8bfd563fa48f-kube-api-access-vfj5g\") pod \"payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c\" (UID: \"f1a597d7-5f58-40a0-82bb-8bfd563fa48f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c"
Jul 28 15:08:02.606556 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.606530    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766"
Jul 28 15:08:02.622570 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.622534    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c"
Jul 28 15:08:02.765088 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.765031    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766"]
Jul 28 15:08:02.768701 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:08:02.768664    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode286c623_e2d5_4206_ad16_3053b95d7ac4.slice/crio-61445e4cc6f243bd88ddf1b5dfcf6d5198482433420295a847126a74539b16c6 WatchSource:0}: Error finding container 61445e4cc6f243bd88ddf1b5dfcf6d5198482433420295a847126a74539b16c6: Status 404 returned error can't find the container with id 61445e4cc6f243bd88ddf1b5dfcf6d5198482433420295a847126a74539b16c6
Jul 28 15:08:02.787401 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:02.787318    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c"]
Jul 28 15:08:02.789435 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:08:02.789408    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1a597d7_5f58_40a0_82bb_8bfd563fa48f.slice/crio-158d09366dc16101e2835718f615a5c0f79218199387da49e50a589b69a40260 WatchSource:0}: Error finding container 158d09366dc16101e2835718f615a5c0f79218199387da49e50a589b69a40260: Status 404 returned error can't find the container with id 158d09366dc16101e2835718f615a5c0f79218199387da49e50a589b69a40260
Jul 28 15:08:03.465392 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:03.465357    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn" event={"ID":"ef57fd68-f7c2-484a-9d80-3dd3a068b7c9","Type":"ContainerStarted","Data":"de5611d7fd2950f9e8ff68b4c19c064e4b37310de0bbe6e6336d1e4137342b02"}
Jul 28 15:08:03.465597 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:03.465468    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"
Jul 28 15:08:03.466891 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:03.466862    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c" event={"ID":"f1a597d7-5f58-40a0-82bb-8bfd563fa48f","Type":"ContainerStarted","Data":"66e97059c86cf0aa42b7aeb11693e1fd1296eeadcb73970684ba99a37323217f"}
Jul 28 15:08:03.467022 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:03.466911    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c" event={"ID":"f1a597d7-5f58-40a0-82bb-8bfd563fa48f","Type":"ContainerStarted","Data":"158d09366dc16101e2835718f615a5c0f79218199387da49e50a589b69a40260"}
Jul 28 15:08:03.467022 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:03.466944    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c"
Jul 28 15:08:03.468304 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:03.468280    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766" event={"ID":"e286c623-e2d5-4206-ad16-3053b95d7ac4","Type":"ContainerStarted","Data":"4462db7eed15b4160212b43bef99baf853f080a9d68cd8cfe854c0a1bf878b4e"}
Jul 28 15:08:03.468401 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:03.468317    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766" event={"ID":"e286c623-e2d5-4206-ad16-3053b95d7ac4","Type":"ContainerStarted","Data":"61445e4cc6f243bd88ddf1b5dfcf6d5198482433420295a847126a74539b16c6"}
Jul 28 15:08:03.468401 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:03.468356    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766"
Jul 28 15:08:03.484507 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:03.484462    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn" podStartSLOduration=1.911171401 podStartE2EDuration="2.484449016s" totalImagesPullingTime="573.277615ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:08:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:08:03.482381587 +0000 UTC m=+2953.283923803" watchObservedRunningTime="2026-07-28 15:08:03.484449016 +0000 UTC m=+2953.285991231"
Jul 28 15:08:03.499397 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:03.499352    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c" podStartSLOduration=1.4993385510000001 podStartE2EDuration="1.499338551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:08:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:08:03.497808168 +0000 UTC m=+2953.299350380" watchObservedRunningTime="2026-07-28 15:08:03.499338551 +0000 UTC m=+2953.300880823"
Jul 28 15:08:03.513460 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:03.513417    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766" podStartSLOduration=1.513404865 podStartE2EDuration="1.513404865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:08:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:08:03.512240797 +0000 UTC m=+2953.313783011" watchObservedRunningTime="2026-07-28 15:08:03.513404865 +0000 UTC m=+2953.314947096"
Jul 28 15:08:09.478567 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:09.478536    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"
Jul 28 15:08:14.475990 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:14.475948    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c"
Jul 28 15:08:14.476463 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:14.476032    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766"
Jul 28 15:08:16.449349 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:16.449310    2584 patch_prober.go:28] interesting pod/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 15:08:16.449730 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:16.449387    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt" podUID="1f12e0d7-1a88-426c-b008-196ba2bd65c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 15:08:17.541356 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:17.541319    2584 generic.go:356] "Generic (PLEG): container finished" podID="1f12e0d7-1a88-426c-b008-196ba2bd65c2" containerID="6350e61297601e9ae95d71e0c9670e0c8834c311754c63db3579c17b757002cf" exitCode=0
Jul 28 15:08:17.541785 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:17.541392    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt" event={"ID":"1f12e0d7-1a88-426c-b008-196ba2bd65c2","Type":"ContainerDied","Data":"6350e61297601e9ae95d71e0c9670e0c8834c311754c63db3579c17b757002cf"}
Jul 28 15:08:17.541785 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:17.541772    2584 scope.go:117] "RemoveContainer" containerID="6350e61297601e9ae95d71e0c9670e0c8834c311754c63db3579c17b757002cf"
Jul 28 15:08:17.542873 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:17.542842    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 15:08:17.542981 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:17.542925    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 15:08:17.542981 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:17.542974    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 15:08:18.553002 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:18.552954    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt" event={"ID":"1f12e0d7-1a88-426c-b008-196ba2bd65c2","Type":"ContainerStarted","Data":"68b7ed077a024e74fb299d0487a1a73d1497e5a3a52ea623807c6c3586659360"}
Jul 28 15:08:19.462435 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:19.462401    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:08:19.462678 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:19.462656    2584 patch_prober.go:28] interesting pod/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 28 15:08:19.462725 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:19.462704    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt" podUID="1f12e0d7-1a88-426c-b008-196ba2bd65c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 28 15:08:20.462037 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:20.462004    2584 patch_prober.go:28] interesting pod/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 28 15:08:20.462471 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:20.462056    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt" podUID="1f12e0d7-1a88-426c-b008-196ba2bd65c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 28 15:08:21.461639 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:21.461605    2584 patch_prober.go:28] interesting pod/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 28 15:08:21.461856 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:21.461665    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt" podUID="1f12e0d7-1a88-426c-b008-196ba2bd65c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 28 15:08:22.462504 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:22.462446    2584 patch_prober.go:28] interesting pod/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 28 15:08:22.463018 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:22.462534    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt" podUID="1f12e0d7-1a88-426c-b008-196ba2bd65c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 28 15:08:23.465487 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:23.465453    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:08:23.465875 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:23.465715    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:08:23.466396 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:08:23.466368    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:10:33.964706 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:33.964660    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"]
Jul 28 15:10:33.965247 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:33.964876    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk" podUID="aeb069e6-12a6-4c8b-a989-adef9d7deabd" containerName="maas-api" containerID="cri-o://534b86911e3f2678506a55c6dabf5597a802501edc0ebd9acefbe1af0446a62a" gracePeriod=30
Jul 28 15:10:34.061086 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.061051    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n"]
Jul 28 15:10:34.061371 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.061323    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n" podUID="74884c15-557b-4ae0-b667-725b998e61d9" containerName="payload-processing" containerID="cri-o://ef3b50e424341eb11abf712e2d31f13cc4f9835cfb46ee8143f3c116d499d0df" gracePeriod=30
Jul 28 15:10:34.079424 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.079323    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn"]
Jul 28 15:10:34.079664 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.079628    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn" podUID="a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f" containerName="payload-pre-processing" containerID="cri-o://317bb58ffb8c6fb85cc55e50d68a3dc01bd66aa0560ac42cd770deef528ef538" gracePeriod=30
Jul 28 15:10:34.236896 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.236849    2584 generic.go:356] "Generic (PLEG): container finished" podID="74884c15-557b-4ae0-b667-725b998e61d9" containerID="ef3b50e424341eb11abf712e2d31f13cc4f9835cfb46ee8143f3c116d499d0df" exitCode=0
Jul 28 15:10:34.237086 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.236938    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n" event={"ID":"74884c15-557b-4ae0-b667-725b998e61d9","Type":"ContainerDied","Data":"ef3b50e424341eb11abf712e2d31f13cc4f9835cfb46ee8143f3c116d499d0df"}
Jul 28 15:10:34.240310 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.239731    2584 generic.go:356] "Generic (PLEG): container finished" podID="a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f" containerID="317bb58ffb8c6fb85cc55e50d68a3dc01bd66aa0560ac42cd770deef528ef538" exitCode=0
Jul 28 15:10:34.240310 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.239865    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn" event={"ID":"a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f","Type":"ContainerDied","Data":"317bb58ffb8c6fb85cc55e50d68a3dc01bd66aa0560ac42cd770deef528ef538"}
Jul 28 15:10:34.241796 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.241646    2584 generic.go:356] "Generic (PLEG): container finished" podID="aeb069e6-12a6-4c8b-a989-adef9d7deabd" containerID="534b86911e3f2678506a55c6dabf5597a802501edc0ebd9acefbe1af0446a62a" exitCode=0
Jul 28 15:10:34.241796 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.241704    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk" event={"ID":"aeb069e6-12a6-4c8b-a989-adef9d7deabd","Type":"ContainerDied","Data":"534b86911e3f2678506a55c6dabf5597a802501edc0ebd9acefbe1af0446a62a"}
Jul 28 15:10:34.241796 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.241747    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk" event={"ID":"aeb069e6-12a6-4c8b-a989-adef9d7deabd","Type":"ContainerDied","Data":"2245ca820286207b1d2ced6e348776cc9088db29d4b7327de87b951dd40753f5"}
Jul 28 15:10:34.241796 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.241769    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2245ca820286207b1d2ced6e348776cc9088db29d4b7327de87b951dd40753f5"
Jul 28 15:10:34.245582 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.245557    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"
Jul 28 15:10:34.321468 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.321396    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aeb069e6-12a6-4c8b-a989-adef9d7deabd-maas-api-tls\") pod \"aeb069e6-12a6-4c8b-a989-adef9d7deabd\" (UID: \"aeb069e6-12a6-4c8b-a989-adef9d7deabd\") "
Jul 28 15:10:34.321684 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.321477    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeb069e6-12a6-4c8b-a989-adef9d7deabd-openshift-service-ca\") pod \"aeb069e6-12a6-4c8b-a989-adef9d7deabd\" (UID: \"aeb069e6-12a6-4c8b-a989-adef9d7deabd\") "
Jul 28 15:10:34.321684 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.321511    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zv5d4\" (UniqueName: \"kubernetes.io/projected/aeb069e6-12a6-4c8b-a989-adef9d7deabd-kube-api-access-zv5d4\") pod \"aeb069e6-12a6-4c8b-a989-adef9d7deabd\" (UID: \"aeb069e6-12a6-4c8b-a989-adef9d7deabd\") "
Jul 28 15:10:34.321940 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.321876    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aeb069e6-12a6-4c8b-a989-adef9d7deabd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aeb069e6-12a6-4c8b-a989-adef9d7deabd" (UID: "aeb069e6-12a6-4c8b-a989-adef9d7deabd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:10:34.324748 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.324718    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aeb069e6-12a6-4c8b-a989-adef9d7deabd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aeb069e6-12a6-4c8b-a989-adef9d7deabd" (UID: "aeb069e6-12a6-4c8b-a989-adef9d7deabd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:10:34.327307 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.327273    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aeb069e6-12a6-4c8b-a989-adef9d7deabd-kube-api-access-zv5d4" (OuterVolumeSpecName: "kube-api-access-zv5d4") pod "aeb069e6-12a6-4c8b-a989-adef9d7deabd" (UID: "aeb069e6-12a6-4c8b-a989-adef9d7deabd"). InnerVolumeSpecName "kube-api-access-zv5d4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:10:34.353904 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.353874    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n"
Jul 28 15:10:34.393172 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.393145    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn"
Jul 28 15:10:34.422952 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.422919    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-776wh\" (UniqueName: \"kubernetes.io/projected/74884c15-557b-4ae0-b667-725b998e61d9-kube-api-access-776wh\") pod \"74884c15-557b-4ae0-b667-725b998e61d9\" (UID: \"74884c15-557b-4ae0-b667-725b998e61d9\") "
Jul 28 15:10:34.423163 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.422976    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74884c15-557b-4ae0-b667-725b998e61d9-plugins-config-volume\") pod \"74884c15-557b-4ae0-b667-725b998e61d9\" (UID: \"74884c15-557b-4ae0-b667-725b998e61d9\") "
Jul 28 15:10:34.423383 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.423354    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74884c15-557b-4ae0-b667-725b998e61d9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "74884c15-557b-4ae0-b667-725b998e61d9" (UID: "74884c15-557b-4ae0-b667-725b998e61d9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:10:34.423383 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.423359    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aeb069e6-12a6-4c8b-a989-adef9d7deabd-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:10:34.423609 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.423410    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeb069e6-12a6-4c8b-a989-adef9d7deabd-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:10:34.423609 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.423423    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zv5d4\" (UniqueName: \"kubernetes.io/projected/aeb069e6-12a6-4c8b-a989-adef9d7deabd-kube-api-access-zv5d4\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:10:34.425146 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.425123    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74884c15-557b-4ae0-b667-725b998e61d9-kube-api-access-776wh" (OuterVolumeSpecName: "kube-api-access-776wh") pod "74884c15-557b-4ae0-b667-725b998e61d9" (UID: "74884c15-557b-4ae0-b667-725b998e61d9"). InnerVolumeSpecName "kube-api-access-776wh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:10:34.523938 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.523847    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8kkwg\" (UniqueName: \"kubernetes.io/projected/a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f-kube-api-access-8kkwg\") pod \"a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f\" (UID: \"a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f\") "
Jul 28 15:10:34.524134 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.523989    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f-plugins-config-volume\") pod \"a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f\" (UID: \"a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f\") "
Jul 28 15:10:34.524227 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.524148    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-776wh\" (UniqueName: \"kubernetes.io/projected/74884c15-557b-4ae0-b667-725b998e61d9-kube-api-access-776wh\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:10:34.524227 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.524160    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74884c15-557b-4ae0-b667-725b998e61d9-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:10:34.524431 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.524405    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f" (UID: "a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:10:34.525856 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.525837    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f-kube-api-access-8kkwg" (OuterVolumeSpecName: "kube-api-access-8kkwg") pod "a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f" (UID: "a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f"). InnerVolumeSpecName "kube-api-access-8kkwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:10:34.625619 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.625583    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:10:34.625619 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:34.625611    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kkwg\" (UniqueName: \"kubernetes.io/projected/a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f-kube-api-access-8kkwg\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:10:35.248374 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:35.248342    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn"
Jul 28 15:10:35.248877 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:35.248377    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn" event={"ID":"a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f","Type":"ContainerDied","Data":"d1d49d7dd57edfa8373e636f17992560b62cd3bb40f8b7312486596022a7ad8c"}
Jul 28 15:10:35.248877 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:35.248448    2584 scope.go:117] "RemoveContainer" containerID="317bb58ffb8c6fb85cc55e50d68a3dc01bd66aa0560ac42cd770deef528ef538"
Jul 28 15:10:35.249901 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:35.249819    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n" event={"ID":"74884c15-557b-4ae0-b667-725b998e61d9","Type":"ContainerDied","Data":"e51c79609fc15cea27867db260d162e544cb57f0cc0cc55334b674d4e4ee3c2f"}
Jul 28 15:10:35.249901 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:35.249840    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n"
Jul 28 15:10:35.249997 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:35.249945    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"
Jul 28 15:10:35.260644 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:35.260620    2584 scope.go:117] "RemoveContainer" containerID="ef3b50e424341eb11abf712e2d31f13cc4f9835cfb46ee8143f3c116d499d0df"
Jul 28 15:10:35.289493 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:35.289462    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn"]
Jul 28 15:10:35.296834 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:35.296808    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-06cd69-6d85bb6584-plskn"]
Jul 28 15:10:35.326439 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:35.326407    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"]
Jul 28 15:10:35.332493 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:35.332460    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-06cd69-67d95cb4b7-8kkrk"]
Jul 28 15:10:35.363784 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:35.363753    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n"]
Jul 28 15:10:35.378174 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:35.378143    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-06cd69-79948d8766-ttp8n"]
Jul 28 15:10:36.796882 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:36.796850    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74884c15-557b-4ae0-b667-725b998e61d9" path="/var/lib/kubelet/pods/74884c15-557b-4ae0-b667-725b998e61d9/volumes"
Jul 28 15:10:36.797319 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:36.797289    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f" path="/var/lib/kubelet/pods/a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f/volumes"
Jul 28 15:10:36.797626 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:36.797612    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aeb069e6-12a6-4c8b-a989-adef9d7deabd" path="/var/lib/kubelet/pods/aeb069e6-12a6-4c8b-a989-adef9d7deabd/volumes"
Jul 28 15:10:53.515186 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.515138    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"]
Jul 28 15:10:53.515676 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.515400    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn" podUID="ef57fd68-f7c2-484a-9d80-3dd3a068b7c9" containerName="maas-api" containerID="cri-o://de5611d7fd2950f9e8ff68b4c19c064e4b37310de0bbe6e6336d1e4137342b02" gracePeriod=30
Jul 28 15:10:53.607619 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.607584    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c"]
Jul 28 15:10:53.607866 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.607829    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c" podUID="f1a597d7-5f58-40a0-82bb-8bfd563fa48f" containerName="payload-processing" containerID="cri-o://66e97059c86cf0aa42b7aeb11693e1fd1296eeadcb73970684ba99a37323217f" gracePeriod=30
Jul 28 15:10:53.624289 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.624252    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766"]
Jul 28 15:10:53.624525 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.624498    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766" podUID="e286c623-e2d5-4206-ad16-3053b95d7ac4" containerName="payload-pre-processing" containerID="cri-o://4462db7eed15b4160212b43bef99baf853f080a9d68cd8cfe854c0a1bf878b4e" gracePeriod=30
Jul 28 15:10:53.881757 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.881722    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"
Jul 28 15:10:53.912516 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.912488    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c"
Jul 28 15:10:53.915652 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.915628    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef57fd68-f7c2-484a-9d80-3dd3a068b7c9-maas-api-tls\") pod \"ef57fd68-f7c2-484a-9d80-3dd3a068b7c9\" (UID: \"ef57fd68-f7c2-484a-9d80-3dd3a068b7c9\") "
Jul 28 15:10:53.915836 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.915677    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfj5g\" (UniqueName: \"kubernetes.io/projected/f1a597d7-5f58-40a0-82bb-8bfd563fa48f-kube-api-access-vfj5g\") pod \"f1a597d7-5f58-40a0-82bb-8bfd563fa48f\" (UID: \"f1a597d7-5f58-40a0-82bb-8bfd563fa48f\") "
Jul 28 15:10:53.915836 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.915727    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef57fd68-f7c2-484a-9d80-3dd3a068b7c9-openshift-service-ca\") pod \"ef57fd68-f7c2-484a-9d80-3dd3a068b7c9\" (UID: \"ef57fd68-f7c2-484a-9d80-3dd3a068b7c9\") "
Jul 28 15:10:53.915836 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.915752    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1a597d7-5f58-40a0-82bb-8bfd563fa48f-plugins-config-volume\") pod \"f1a597d7-5f58-40a0-82bb-8bfd563fa48f\" (UID: \"f1a597d7-5f58-40a0-82bb-8bfd563fa48f\") "
Jul 28 15:10:53.915836 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.915802    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hxlc\" (UniqueName: \"kubernetes.io/projected/ef57fd68-f7c2-484a-9d80-3dd3a068b7c9-kube-api-access-6hxlc\") pod \"ef57fd68-f7c2-484a-9d80-3dd3a068b7c9\" (UID: \"ef57fd68-f7c2-484a-9d80-3dd3a068b7c9\") "
Jul 28 15:10:53.916329 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.916166    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef57fd68-f7c2-484a-9d80-3dd3a068b7c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ef57fd68-f7c2-484a-9d80-3dd3a068b7c9" (UID: "ef57fd68-f7c2-484a-9d80-3dd3a068b7c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:10:53.916329 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.916250    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef57fd68-f7c2-484a-9d80-3dd3a068b7c9-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:10:53.916329 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.916286    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1a597d7-5f58-40a0-82bb-8bfd563fa48f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f1a597d7-5f58-40a0-82bb-8bfd563fa48f" (UID: "f1a597d7-5f58-40a0-82bb-8bfd563fa48f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:10:53.918139 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.918115    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef57fd68-f7c2-484a-9d80-3dd3a068b7c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ef57fd68-f7c2-484a-9d80-3dd3a068b7c9" (UID: "ef57fd68-f7c2-484a-9d80-3dd3a068b7c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:10:53.918252 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.918214    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef57fd68-f7c2-484a-9d80-3dd3a068b7c9-kube-api-access-6hxlc" (OuterVolumeSpecName: "kube-api-access-6hxlc") pod "ef57fd68-f7c2-484a-9d80-3dd3a068b7c9" (UID: "ef57fd68-f7c2-484a-9d80-3dd3a068b7c9"). InnerVolumeSpecName "kube-api-access-6hxlc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:10:53.918320 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.918290    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1a597d7-5f58-40a0-82bb-8bfd563fa48f-kube-api-access-vfj5g" (OuterVolumeSpecName: "kube-api-access-vfj5g") pod "f1a597d7-5f58-40a0-82bb-8bfd563fa48f" (UID: "f1a597d7-5f58-40a0-82bb-8bfd563fa48f"). InnerVolumeSpecName "kube-api-access-vfj5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:10:53.932215 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:53.932176    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766"
Jul 28 15:10:54.016635 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.016592    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgxmg\" (UniqueName: \"kubernetes.io/projected/e286c623-e2d5-4206-ad16-3053b95d7ac4-kube-api-access-tgxmg\") pod \"e286c623-e2d5-4206-ad16-3053b95d7ac4\" (UID: \"e286c623-e2d5-4206-ad16-3053b95d7ac4\") "
Jul 28 15:10:54.016824 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.016660    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e286c623-e2d5-4206-ad16-3053b95d7ac4-plugins-config-volume\") pod \"e286c623-e2d5-4206-ad16-3053b95d7ac4\" (UID: \"e286c623-e2d5-4206-ad16-3053b95d7ac4\") "
Jul 28 15:10:54.016891 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.016872    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef57fd68-f7c2-484a-9d80-3dd3a068b7c9-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:10:54.016928 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.016887    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfj5g\" (UniqueName: \"kubernetes.io/projected/f1a597d7-5f58-40a0-82bb-8bfd563fa48f-kube-api-access-vfj5g\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:10:54.016928 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.016900    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1a597d7-5f58-40a0-82bb-8bfd563fa48f-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:10:54.016928 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.016909    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hxlc\" (UniqueName: \"kubernetes.io/projected/ef57fd68-f7c2-484a-9d80-3dd3a068b7c9-kube-api-access-6hxlc\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:10:54.017030 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.016994    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e286c623-e2d5-4206-ad16-3053b95d7ac4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e286c623-e2d5-4206-ad16-3053b95d7ac4" (UID: "e286c623-e2d5-4206-ad16-3053b95d7ac4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:10:54.018632 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.018612    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e286c623-e2d5-4206-ad16-3053b95d7ac4-kube-api-access-tgxmg" (OuterVolumeSpecName: "kube-api-access-tgxmg") pod "e286c623-e2d5-4206-ad16-3053b95d7ac4" (UID: "e286c623-e2d5-4206-ad16-3053b95d7ac4"). InnerVolumeSpecName "kube-api-access-tgxmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:10:54.117477 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.117447    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgxmg\" (UniqueName: \"kubernetes.io/projected/e286c623-e2d5-4206-ad16-3053b95d7ac4-kube-api-access-tgxmg\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:10:54.117477 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.117475    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e286c623-e2d5-4206-ad16-3053b95d7ac4-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:10:54.342492 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.342454    2584 generic.go:356] "Generic (PLEG): container finished" podID="ef57fd68-f7c2-484a-9d80-3dd3a068b7c9" containerID="de5611d7fd2950f9e8ff68b4c19c064e4b37310de0bbe6e6336d1e4137342b02" exitCode=0
Jul 28 15:10:54.342704 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.342501    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"
Jul 28 15:10:54.342704 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.342545    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn" event={"ID":"ef57fd68-f7c2-484a-9d80-3dd3a068b7c9","Type":"ContainerDied","Data":"de5611d7fd2950f9e8ff68b4c19c064e4b37310de0bbe6e6336d1e4137342b02"}
Jul 28 15:10:54.342704 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.342611    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn" event={"ID":"ef57fd68-f7c2-484a-9d80-3dd3a068b7c9","Type":"ContainerDied","Data":"781c8ea2f2dd560cec1a1c6829728516bf0b87ed27b1b9bcbf295afe3c35b8cd"}
Jul 28 15:10:54.342704 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.342643    2584 scope.go:117] "RemoveContainer" containerID="de5611d7fd2950f9e8ff68b4c19c064e4b37310de0bbe6e6336d1e4137342b02"
Jul 28 15:10:54.343875 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.343858    2584 generic.go:356] "Generic (PLEG): container finished" podID="f1a597d7-5f58-40a0-82bb-8bfd563fa48f" containerID="66e97059c86cf0aa42b7aeb11693e1fd1296eeadcb73970684ba99a37323217f" exitCode=0
Jul 28 15:10:54.343964 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.343903    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c"
Jul 28 15:10:54.344019 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.343949    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c" event={"ID":"f1a597d7-5f58-40a0-82bb-8bfd563fa48f","Type":"ContainerDied","Data":"66e97059c86cf0aa42b7aeb11693e1fd1296eeadcb73970684ba99a37323217f"}
Jul 28 15:10:54.344019 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.344002    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c" event={"ID":"f1a597d7-5f58-40a0-82bb-8bfd563fa48f","Type":"ContainerDied","Data":"158d09366dc16101e2835718f615a5c0f79218199387da49e50a589b69a40260"}
Jul 28 15:10:54.345557 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.345529    2584 generic.go:356] "Generic (PLEG): container finished" podID="e286c623-e2d5-4206-ad16-3053b95d7ac4" containerID="4462db7eed15b4160212b43bef99baf853f080a9d68cd8cfe854c0a1bf878b4e" exitCode=0
Jul 28 15:10:54.345689 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.345573    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766"
Jul 28 15:10:54.345689 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.345582    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766" event={"ID":"e286c623-e2d5-4206-ad16-3053b95d7ac4","Type":"ContainerDied","Data":"4462db7eed15b4160212b43bef99baf853f080a9d68cd8cfe854c0a1bf878b4e"}
Jul 28 15:10:54.345689 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.345633    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766" event={"ID":"e286c623-e2d5-4206-ad16-3053b95d7ac4","Type":"ContainerDied","Data":"61445e4cc6f243bd88ddf1b5dfcf6d5198482433420295a847126a74539b16c6"}
Jul 28 15:10:54.353186 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.353167    2584 scope.go:117] "RemoveContainer" containerID="de5611d7fd2950f9e8ff68b4c19c064e4b37310de0bbe6e6336d1e4137342b02"
Jul 28 15:10:54.353483 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:10:54.353461    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de5611d7fd2950f9e8ff68b4c19c064e4b37310de0bbe6e6336d1e4137342b02\": container with ID starting with de5611d7fd2950f9e8ff68b4c19c064e4b37310de0bbe6e6336d1e4137342b02 not found: ID does not exist" containerID="de5611d7fd2950f9e8ff68b4c19c064e4b37310de0bbe6e6336d1e4137342b02"
Jul 28 15:10:54.353586 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.353492    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de5611d7fd2950f9e8ff68b4c19c064e4b37310de0bbe6e6336d1e4137342b02"} err="failed to get container status \"de5611d7fd2950f9e8ff68b4c19c064e4b37310de0bbe6e6336d1e4137342b02\": rpc error: code = NotFound desc = could not find container \"de5611d7fd2950f9e8ff68b4c19c064e4b37310de0bbe6e6336d1e4137342b02\": container with ID starting with de5611d7fd2950f9e8ff68b4c19c064e4b37310de0bbe6e6336d1e4137342b02 not found: ID does not exist"
Jul 28 15:10:54.353586 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.353516    2584 scope.go:117] "RemoveContainer" containerID="66e97059c86cf0aa42b7aeb11693e1fd1296eeadcb73970684ba99a37323217f"
Jul 28 15:10:54.361948 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.361929    2584 scope.go:117] "RemoveContainer" containerID="66e97059c86cf0aa42b7aeb11693e1fd1296eeadcb73970684ba99a37323217f"
Jul 28 15:10:54.362270 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:10:54.362247    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66e97059c86cf0aa42b7aeb11693e1fd1296eeadcb73970684ba99a37323217f\": container with ID starting with 66e97059c86cf0aa42b7aeb11693e1fd1296eeadcb73970684ba99a37323217f not found: ID does not exist" containerID="66e97059c86cf0aa42b7aeb11693e1fd1296eeadcb73970684ba99a37323217f"
Jul 28 15:10:54.362323 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.362281    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66e97059c86cf0aa42b7aeb11693e1fd1296eeadcb73970684ba99a37323217f"} err="failed to get container status \"66e97059c86cf0aa42b7aeb11693e1fd1296eeadcb73970684ba99a37323217f\": rpc error: code = NotFound desc = could not find container \"66e97059c86cf0aa42b7aeb11693e1fd1296eeadcb73970684ba99a37323217f\": container with ID starting with 66e97059c86cf0aa42b7aeb11693e1fd1296eeadcb73970684ba99a37323217f not found: ID does not exist"
Jul 28 15:10:54.362323 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.362302    2584 scope.go:117] "RemoveContainer" containerID="4462db7eed15b4160212b43bef99baf853f080a9d68cd8cfe854c0a1bf878b4e"
Jul 28 15:10:54.369926 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.369906    2584 scope.go:117] "RemoveContainer" containerID="4462db7eed15b4160212b43bef99baf853f080a9d68cd8cfe854c0a1bf878b4e"
Jul 28 15:10:54.370208 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:10:54.370179    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4462db7eed15b4160212b43bef99baf853f080a9d68cd8cfe854c0a1bf878b4e\": container with ID starting with 4462db7eed15b4160212b43bef99baf853f080a9d68cd8cfe854c0a1bf878b4e not found: ID does not exist" containerID="4462db7eed15b4160212b43bef99baf853f080a9d68cd8cfe854c0a1bf878b4e"
Jul 28 15:10:54.370295 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.370209    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4462db7eed15b4160212b43bef99baf853f080a9d68cd8cfe854c0a1bf878b4e"} err="failed to get container status \"4462db7eed15b4160212b43bef99baf853f080a9d68cd8cfe854c0a1bf878b4e\": rpc error: code = NotFound desc = could not find container \"4462db7eed15b4160212b43bef99baf853f080a9d68cd8cfe854c0a1bf878b4e\": container with ID starting with 4462db7eed15b4160212b43bef99baf853f080a9d68cd8cfe854c0a1bf878b4e not found: ID does not exist"
Jul 28 15:10:54.395263 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.395229    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766"]
Jul 28 15:10:54.400767 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.400739    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f4853e-c4b97cf56-cf766"]
Jul 28 15:10:54.435102 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.435052    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c"]
Jul 28 15:10:54.446445 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.446414    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f4853e-6669597558-ptn4c"]
Jul 28 15:10:54.475610 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.475575    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"]
Jul 28 15:10:54.486819 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.486783    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f4853e-fb59b576f-zrbsn"]
Jul 28 15:10:54.796746 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.796665    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e286c623-e2d5-4206-ad16-3053b95d7ac4" path="/var/lib/kubelet/pods/e286c623-e2d5-4206-ad16-3053b95d7ac4/volumes"
Jul 28 15:10:54.797124 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.797067    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef57fd68-f7c2-484a-9d80-3dd3a068b7c9" path="/var/lib/kubelet/pods/ef57fd68-f7c2-484a-9d80-3dd3a068b7c9/volumes"
Jul 28 15:10:54.797389 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:10:54.797376    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1a597d7-5f58-40a0-82bb-8bfd563fa48f" path="/var/lib/kubelet/pods/f1a597d7-5f58-40a0-82bb-8bfd563fa48f/volumes"
Jul 28 15:11:01.306566 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:01.306533    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"]
Jul 28 15:11:01.306984 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:01.306758    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt" podUID="1f12e0d7-1a88-426c-b008-196ba2bd65c2" containerName="istio-proxy" containerID="cri-o://68b7ed077a024e74fb299d0487a1a73d1497e5a3a52ea623807c6c3586659360" gracePeriod=30
Jul 28 15:11:05.928587 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.928552    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"]
Jul 28 15:11:05.929080 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.928912    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeb069e6-12a6-4c8b-a989-adef9d7deabd" containerName="maas-api"
Jul 28 15:11:05.929080 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.928923    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeb069e6-12a6-4c8b-a989-adef9d7deabd" containerName="maas-api"
Jul 28 15:11:05.929080 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.928934    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74884c15-557b-4ae0-b667-725b998e61d9" containerName="payload-processing"
Jul 28 15:11:05.929080 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.928940    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="74884c15-557b-4ae0-b667-725b998e61d9" containerName="payload-processing"
Jul 28 15:11:05.929080 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.928951    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f" containerName="payload-pre-processing"
Jul 28 15:11:05.929080 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.928957    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f" containerName="payload-pre-processing"
Jul 28 15:11:05.929080 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.928973    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef57fd68-f7c2-484a-9d80-3dd3a068b7c9" containerName="maas-api"
Jul 28 15:11:05.929080 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.928978    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef57fd68-f7c2-484a-9d80-3dd3a068b7c9" containerName="maas-api"
Jul 28 15:11:05.929080 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.928984    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e286c623-e2d5-4206-ad16-3053b95d7ac4" containerName="payload-pre-processing"
Jul 28 15:11:05.929080 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.928989    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e286c623-e2d5-4206-ad16-3053b95d7ac4" containerName="payload-pre-processing"
Jul 28 15:11:05.929080 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.928994    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1a597d7-5f58-40a0-82bb-8bfd563fa48f" containerName="payload-processing"
Jul 28 15:11:05.929080 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.928999    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1a597d7-5f58-40a0-82bb-8bfd563fa48f" containerName="payload-processing"
Jul 28 15:11:05.929080 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.929051    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a739e9e9-acb6-4aa6-bb00-bc8bf6a6e56f" containerName="payload-pre-processing"
Jul 28 15:11:05.929080 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.929059    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef57fd68-f7c2-484a-9d80-3dd3a068b7c9" containerName="maas-api"
Jul 28 15:11:05.929080 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.929066    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e286c623-e2d5-4206-ad16-3053b95d7ac4" containerName="payload-pre-processing"
Jul 28 15:11:05.929080 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.929072    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1a597d7-5f58-40a0-82bb-8bfd563fa48f" containerName="payload-processing"
Jul 28 15:11:05.929080 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.929077    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeb069e6-12a6-4c8b-a989-adef9d7deabd" containerName="maas-api"
Jul 28 15:11:05.929080 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.929083    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="74884c15-557b-4ae0-b667-725b998e61d9" containerName="payload-processing"
Jul 28 15:11:05.931808 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.931785    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"
Jul 28 15:11:05.934434 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.934411    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-cef5a5-serving-cert\""
Jul 28 15:11:05.953239 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:05.953202    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"]
Jul 28 15:11:06.016788 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.016752    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqc75\" (UniqueName: \"kubernetes.io/projected/d078f02c-9add-4f94-80a6-01830b0e4d5f-kube-api-access-nqc75\") pod \"maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7\" (UID: \"d078f02c-9add-4f94-80a6-01830b0e4d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"
Jul 28 15:11:06.016995 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.016819    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d078f02c-9add-4f94-80a6-01830b0e4d5f-openshift-service-ca\") pod \"maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7\" (UID: \"d078f02c-9add-4f94-80a6-01830b0e4d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"
Jul 28 15:11:06.016995 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.016935    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d078f02c-9add-4f94-80a6-01830b0e4d5f-maas-api-tls\") pod \"maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7\" (UID: \"d078f02c-9add-4f94-80a6-01830b0e4d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"
Jul 28 15:11:06.117663 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.117621    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqc75\" (UniqueName: \"kubernetes.io/projected/d078f02c-9add-4f94-80a6-01830b0e4d5f-kube-api-access-nqc75\") pod \"maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7\" (UID: \"d078f02c-9add-4f94-80a6-01830b0e4d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"
Jul 28 15:11:06.117873 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.117674    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d078f02c-9add-4f94-80a6-01830b0e4d5f-openshift-service-ca\") pod \"maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7\" (UID: \"d078f02c-9add-4f94-80a6-01830b0e4d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"
Jul 28 15:11:06.117873 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.117728    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d078f02c-9add-4f94-80a6-01830b0e4d5f-maas-api-tls\") pod \"maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7\" (UID: \"d078f02c-9add-4f94-80a6-01830b0e4d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"
Jul 28 15:11:06.118546 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.118521    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d078f02c-9add-4f94-80a6-01830b0e4d5f-openshift-service-ca\") pod \"maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7\" (UID: \"d078f02c-9add-4f94-80a6-01830b0e4d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"
Jul 28 15:11:06.120318 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.120299    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d078f02c-9add-4f94-80a6-01830b0e4d5f-maas-api-tls\") pod \"maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7\" (UID: \"d078f02c-9add-4f94-80a6-01830b0e4d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"
Jul 28 15:11:06.127786 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.127760    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqc75\" (UniqueName: \"kubernetes.io/projected/d078f02c-9add-4f94-80a6-01830b0e4d5f-kube-api-access-nqc75\") pod \"maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7\" (UID: \"d078f02c-9add-4f94-80a6-01830b0e4d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"
Jul 28 15:11:06.242640 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.242543    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"
Jul 28 15:11:06.404747 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.404701    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"]
Jul 28 15:11:06.405773 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:11:06.405750    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd078f02c_9add_4f94_80a6_01830b0e4d5f.slice/crio-6ff26b64e402b1d179618696251305f975c1391cb8f09108a39403ee9ce01809 WatchSource:0}: Error finding container 6ff26b64e402b1d179618696251305f975c1391cb8f09108a39403ee9ce01809: Status 404 returned error can't find the container with id 6ff26b64e402b1d179618696251305f975c1391cb8f09108a39403ee9ce01809
Jul 28 15:11:06.559268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.559229    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:11:06.621885 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.621853    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5qm4\" (UniqueName: \"kubernetes.io/projected/1f12e0d7-1a88-426c-b008-196ba2bd65c2-kube-api-access-n5qm4\") pod \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") "
Jul 28 15:11:06.622112 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.621954    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-credential-socket\") pod \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") "
Jul 28 15:11:06.622112 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.622045    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-envoy\") pod \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") "
Jul 28 15:11:06.622112 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.622073    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-token\") pod \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") "
Jul 28 15:11:06.622112 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.622108    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-workload-socket\") pod \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") "
Jul 28 15:11:06.622335 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.622147    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-workload-certs\") pod \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") "
Jul 28 15:11:06.622335 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.622173    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-podinfo\") pod \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") "
Jul 28 15:11:06.622335 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.622221    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-data\") pod \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") "
Jul 28 15:11:06.622335 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.622250    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istiod-ca-cert\") pod \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\" (UID: \"1f12e0d7-1a88-426c-b008-196ba2bd65c2\") "
Jul 28 15:11:06.622539 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.622394    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1f12e0d7-1a88-426c-b008-196ba2bd65c2" (UID: "1f12e0d7-1a88-426c-b008-196ba2bd65c2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:11:06.622539 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.622449    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1f12e0d7-1a88-426c-b008-196ba2bd65c2" (UID: "1f12e0d7-1a88-426c-b008-196ba2bd65c2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:11:06.622856 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.622659    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-workload-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:11:06.622856 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.622685    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-credential-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:11:06.622856 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.622783    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1f12e0d7-1a88-426c-b008-196ba2bd65c2" (UID: "1f12e0d7-1a88-426c-b008-196ba2bd65c2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:11:06.623217 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.623186    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1f12e0d7-1a88-426c-b008-196ba2bd65c2" (UID: "1f12e0d7-1a88-426c-b008-196ba2bd65c2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:11:06.623406 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.623380    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-data" (OuterVolumeSpecName: "istio-data") pod "1f12e0d7-1a88-426c-b008-196ba2bd65c2" (UID: "1f12e0d7-1a88-426c-b008-196ba2bd65c2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:11:06.624527 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.624498    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f12e0d7-1a88-426c-b008-196ba2bd65c2-kube-api-access-n5qm4" (OuterVolumeSpecName: "kube-api-access-n5qm4") pod "1f12e0d7-1a88-426c-b008-196ba2bd65c2" (UID: "1f12e0d7-1a88-426c-b008-196ba2bd65c2"). InnerVolumeSpecName "kube-api-access-n5qm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:11:06.624527 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.624495    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1f12e0d7-1a88-426c-b008-196ba2bd65c2" (UID: "1f12e0d7-1a88-426c-b008-196ba2bd65c2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 15:11:06.624690 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.624672    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1f12e0d7-1a88-426c-b008-196ba2bd65c2" (UID: "1f12e0d7-1a88-426c-b008-196ba2bd65c2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:11:06.624784 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.624761    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-token" (OuterVolumeSpecName: "istio-token") pod "1f12e0d7-1a88-426c-b008-196ba2bd65c2" (UID: "1f12e0d7-1a88-426c-b008-196ba2bd65c2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:11:06.723931 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.723890    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-envoy\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:11:06.723931 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.723925    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-token\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:11:06.723931 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.723934    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-workload-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:11:06.723931 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.723944    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-podinfo\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:11:06.724259 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.723953    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istio-data\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:11:06.724259 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.723961    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f12e0d7-1a88-426c-b008-196ba2bd65c2-istiod-ca-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:11:06.724259 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:06.723969    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5qm4\" (UniqueName: \"kubernetes.io/projected/1f12e0d7-1a88-426c-b008-196ba2bd65c2-kube-api-access-n5qm4\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:11:07.018132 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.018081    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h"]
Jul 28 15:11:07.018648 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.018628    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f12e0d7-1a88-426c-b008-196ba2bd65c2" containerName="istio-proxy"
Jul 28 15:11:07.018726 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.018650    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f12e0d7-1a88-426c-b008-196ba2bd65c2" containerName="istio-proxy"
Jul 28 15:11:07.018726 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.018684    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f12e0d7-1a88-426c-b008-196ba2bd65c2" containerName="istio-proxy"
Jul 28 15:11:07.018726 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.018693    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f12e0d7-1a88-426c-b008-196ba2bd65c2" containerName="istio-proxy"
Jul 28 15:11:07.018889 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.018783    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f12e0d7-1a88-426c-b008-196ba2bd65c2" containerName="istio-proxy"
Jul 28 15:11:07.023818 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.023799    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h"
Jul 28 15:11:07.028441 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.028418    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-cef5a5\""
Jul 28 15:11:07.029760 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.029740    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-cef5a5-dockercfg-4hdgp\""
Jul 28 15:11:07.038122 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.038065    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h"]
Jul 28 15:11:07.089756 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.089667    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v"]
Jul 28 15:11:07.090147 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.090132    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f12e0d7-1a88-426c-b008-196ba2bd65c2" containerName="istio-proxy"
Jul 28 15:11:07.093237 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.093222    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v"
Jul 28 15:11:07.112216 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.112177    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v"]
Jul 28 15:11:07.128697 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.128530    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fszd\" (UniqueName: \"kubernetes.io/projected/a7ba30f1-a2c2-4484-b6f3-4873b304876b-kube-api-access-6fszd\") pod \"payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h\" (UID: \"a7ba30f1-a2c2-4484-b6f3-4873b304876b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h"
Jul 28 15:11:07.128697 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.128589    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7ba30f1-a2c2-4484-b6f3-4873b304876b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h\" (UID: \"a7ba30f1-a2c2-4484-b6f3-4873b304876b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h"
Jul 28 15:11:07.128697 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.128668    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a135e30-9600-4086-9000-79683e9e4dad-plugins-config-volume\") pod \"payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v\" (UID: \"1a135e30-9600-4086-9000-79683e9e4dad\") " pod="openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v"
Jul 28 15:11:07.129025 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.128739    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfcgx\" (UniqueName: \"kubernetes.io/projected/1a135e30-9600-4086-9000-79683e9e4dad-kube-api-access-dfcgx\") pod \"payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v\" (UID: \"1a135e30-9600-4086-9000-79683e9e4dad\") " pod="openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v"
Jul 28 15:11:07.230208 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.230172    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a135e30-9600-4086-9000-79683e9e4dad-plugins-config-volume\") pod \"payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v\" (UID: \"1a135e30-9600-4086-9000-79683e9e4dad\") " pod="openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v"
Jul 28 15:11:07.230423 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.230328    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfcgx\" (UniqueName: \"kubernetes.io/projected/1a135e30-9600-4086-9000-79683e9e4dad-kube-api-access-dfcgx\") pod \"payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v\" (UID: \"1a135e30-9600-4086-9000-79683e9e4dad\") " pod="openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v"
Jul 28 15:11:07.230423 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.230401    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fszd\" (UniqueName: \"kubernetes.io/projected/a7ba30f1-a2c2-4484-b6f3-4873b304876b-kube-api-access-6fszd\") pod \"payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h\" (UID: \"a7ba30f1-a2c2-4484-b6f3-4873b304876b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h"
Jul 28 15:11:07.230558 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.230437    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7ba30f1-a2c2-4484-b6f3-4873b304876b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h\" (UID: \"a7ba30f1-a2c2-4484-b6f3-4873b304876b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h"
Jul 28 15:11:07.231191 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.231165    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7ba30f1-a2c2-4484-b6f3-4873b304876b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h\" (UID: \"a7ba30f1-a2c2-4484-b6f3-4873b304876b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h"
Jul 28 15:11:07.231296 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.231169    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a135e30-9600-4086-9000-79683e9e4dad-plugins-config-volume\") pod \"payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v\" (UID: \"1a135e30-9600-4086-9000-79683e9e4dad\") " pod="openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v"
Jul 28 15:11:07.239694 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.239661    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fszd\" (UniqueName: \"kubernetes.io/projected/a7ba30f1-a2c2-4484-b6f3-4873b304876b-kube-api-access-6fszd\") pod \"payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h\" (UID: \"a7ba30f1-a2c2-4484-b6f3-4873b304876b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h"
Jul 28 15:11:07.239871 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.239786    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfcgx\" (UniqueName: \"kubernetes.io/projected/1a135e30-9600-4086-9000-79683e9e4dad-kube-api-access-dfcgx\") pod \"payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v\" (UID: \"1a135e30-9600-4086-9000-79683e9e4dad\") " pod="openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v"
Jul 28 15:11:07.334960 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.334922    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h"
Jul 28 15:11:07.403864 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.403614    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v"
Jul 28 15:11:07.422183 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.422154    2584 generic.go:356] "Generic (PLEG): container finished" podID="1f12e0d7-1a88-426c-b008-196ba2bd65c2" containerID="68b7ed077a024e74fb299d0487a1a73d1497e5a3a52ea623807c6c3586659360" exitCode=0
Jul 28 15:11:07.422443 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.422312    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt" event={"ID":"1f12e0d7-1a88-426c-b008-196ba2bd65c2","Type":"ContainerDied","Data":"68b7ed077a024e74fb299d0487a1a73d1497e5a3a52ea623807c6c3586659360"}
Jul 28 15:11:07.422443 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.422437    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt" event={"ID":"1f12e0d7-1a88-426c-b008-196ba2bd65c2","Type":"ContainerDied","Data":"acba3f0b9195a5fc49132c8e988d26ed50a6caf6cf9536f7a870399c68389a1d"}
Jul 28 15:11:07.422622 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.422448    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"
Jul 28 15:11:07.422622 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.422472    2584 scope.go:117] "RemoveContainer" containerID="68b7ed077a024e74fb299d0487a1a73d1497e5a3a52ea623807c6c3586659360"
Jul 28 15:11:07.425150 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.425030    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7" event={"ID":"d078f02c-9add-4f94-80a6-01830b0e4d5f","Type":"ContainerStarted","Data":"50b6c7bfc877fb9ae1f202aed95464e1fc6ad7c173bbd8e876db981a47ee52e5"}
Jul 28 15:11:07.425150 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.425082    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7" event={"ID":"d078f02c-9add-4f94-80a6-01830b0e4d5f","Type":"ContainerStarted","Data":"6ff26b64e402b1d179618696251305f975c1391cb8f09108a39403ee9ce01809"}
Jul 28 15:11:07.425312 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.425233    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"
Jul 28 15:11:07.432845 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.432824    2584 scope.go:117] "RemoveContainer" containerID="6350e61297601e9ae95d71e0c9670e0c8834c311754c63db3579c17b757002cf"
Jul 28 15:11:07.442173 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.442150    2584 scope.go:117] "RemoveContainer" containerID="68b7ed077a024e74fb299d0487a1a73d1497e5a3a52ea623807c6c3586659360"
Jul 28 15:11:07.442512 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:11:07.442486    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68b7ed077a024e74fb299d0487a1a73d1497e5a3a52ea623807c6c3586659360\": container with ID starting with 68b7ed077a024e74fb299d0487a1a73d1497e5a3a52ea623807c6c3586659360 not found: ID does not exist" containerID="68b7ed077a024e74fb299d0487a1a73d1497e5a3a52ea623807c6c3586659360"
Jul 28 15:11:07.442629 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.442520    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68b7ed077a024e74fb299d0487a1a73d1497e5a3a52ea623807c6c3586659360"} err="failed to get container status \"68b7ed077a024e74fb299d0487a1a73d1497e5a3a52ea623807c6c3586659360\": rpc error: code = NotFound desc = could not find container \"68b7ed077a024e74fb299d0487a1a73d1497e5a3a52ea623807c6c3586659360\": container with ID starting with 68b7ed077a024e74fb299d0487a1a73d1497e5a3a52ea623807c6c3586659360 not found: ID does not exist"
Jul 28 15:11:07.442629 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.442539    2584 scope.go:117] "RemoveContainer" containerID="6350e61297601e9ae95d71e0c9670e0c8834c311754c63db3579c17b757002cf"
Jul 28 15:11:07.442833 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:11:07.442814    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6350e61297601e9ae95d71e0c9670e0c8834c311754c63db3579c17b757002cf\": container with ID starting with 6350e61297601e9ae95d71e0c9670e0c8834c311754c63db3579c17b757002cf not found: ID does not exist" containerID="6350e61297601e9ae95d71e0c9670e0c8834c311754c63db3579c17b757002cf"
Jul 28 15:11:07.442904 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.442837    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6350e61297601e9ae95d71e0c9670e0c8834c311754c63db3579c17b757002cf"} err="failed to get container status \"6350e61297601e9ae95d71e0c9670e0c8834c311754c63db3579c17b757002cf\": rpc error: code = NotFound desc = could not find container \"6350e61297601e9ae95d71e0c9670e0c8834c311754c63db3579c17b757002cf\": container with ID starting with 6350e61297601e9ae95d71e0c9670e0c8834c311754c63db3579c17b757002cf not found: ID does not exist"
Jul 28 15:11:07.460653 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.460583    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7" podStartSLOduration=1.900795526 podStartE2EDuration="2.460559987s" totalImagesPullingTime="559.764461ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:11:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:11:07.458598544 +0000 UTC m=+3137.260140761" watchObservedRunningTime="2026-07-28 15:11:07.460559987 +0000 UTC m=+3137.262102200"
Jul 28 15:11:07.481899 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.481842    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"]
Jul 28 15:11:07.488243 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.488187    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-f4853e-openshift-default-785866c94b-t7tzt"]
Jul 28 15:11:07.510885 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.510857    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h"]
Jul 28 15:11:07.512247 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:11:07.512214    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7ba30f1_a2c2_4484_b6f3_4873b304876b.slice/crio-911ad10f99ad466704869cfd83a9ee0cff68991c74dc93c4fc8940064d032319 WatchSource:0}: Error finding container 911ad10f99ad466704869cfd83a9ee0cff68991c74dc93c4fc8940064d032319: Status 404 returned error can't find the container with id 911ad10f99ad466704869cfd83a9ee0cff68991c74dc93c4fc8940064d032319
Jul 28 15:11:07.578222 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:07.578144    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v"]
Jul 28 15:11:07.581404 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:11:07.581376    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a135e30_9600_4086_9000_79683e9e4dad.slice/crio-2ea68ed56ce29d2234b7f4491f3bb06c62a636ffb3c6d5e5ebe4bd339dc1586e WatchSource:0}: Error finding container 2ea68ed56ce29d2234b7f4491f3bb06c62a636ffb3c6d5e5ebe4bd339dc1586e: Status 404 returned error can't find the container with id 2ea68ed56ce29d2234b7f4491f3bb06c62a636ffb3c6d5e5ebe4bd339dc1586e
Jul 28 15:11:08.433212 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:08.433177    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h" event={"ID":"a7ba30f1-a2c2-4484-b6f3-4873b304876b","Type":"ContainerStarted","Data":"040905e3991c23335a2599bc53b5b60b75a04fbf7b77cc72fc53ab253099a418"}
Jul 28 15:11:08.433706 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:08.433235    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h" event={"ID":"a7ba30f1-a2c2-4484-b6f3-4873b304876b","Type":"ContainerStarted","Data":"911ad10f99ad466704869cfd83a9ee0cff68991c74dc93c4fc8940064d032319"}
Jul 28 15:11:08.433706 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:08.433263    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h"
Jul 28 15:11:08.434839 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:08.434808    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v" event={"ID":"1a135e30-9600-4086-9000-79683e9e4dad","Type":"ContainerStarted","Data":"28babe247be6fe4ec50cc29f9ba426d7ca88f7259c5df19ed09ca66897258a61"}
Jul 28 15:11:08.435062 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:08.435041    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v"
Jul 28 15:11:08.435143 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:08.435081    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v" event={"ID":"1a135e30-9600-4086-9000-79683e9e4dad","Type":"ContainerStarted","Data":"2ea68ed56ce29d2234b7f4491f3bb06c62a636ffb3c6d5e5ebe4bd339dc1586e"}
Jul 28 15:11:08.464623 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:08.464563    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h" podStartSLOduration=2.464545591 podStartE2EDuration="2.464545591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:11:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:11:08.462841493 +0000 UTC m=+3138.264383711" watchObservedRunningTime="2026-07-28 15:11:08.464545591 +0000 UTC m=+3138.266087806"
Jul 28 15:11:08.485920 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:08.485864    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v" podStartSLOduration=1.485847273 podStartE2EDuration="1.485847273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:11:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:11:08.48525342 +0000 UTC m=+3138.286795635" watchObservedRunningTime="2026-07-28 15:11:08.485847273 +0000 UTC m=+3138.287389487"
Jul 28 15:11:08.797600 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:08.797518    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f12e0d7-1a88-426c-b008-196ba2bd65c2" path="/var/lib/kubelet/pods/1f12e0d7-1a88-426c-b008-196ba2bd65c2/volumes"
Jul 28 15:11:13.440967 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:13.440886    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"
Jul 28 15:11:19.442843 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:19.442803    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v"
Jul 28 15:11:19.443302 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:19.443252    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h"
Jul 28 15:11:31.684917 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.684883    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"]
Jul 28 15:11:31.688523 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.688502    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.691061 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.691039    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-becc22-openshift-default-dockercfg-957g6\""
Jul 28 15:11:31.706654 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.706625    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"]
Jul 28 15:11:31.758543 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.758504    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-data\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.758816 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.758787    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-token\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.758945 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.758922    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-podinfo\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.759075 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.759056    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-workload-certs\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.759199 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.759119    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffxmf\" (UniqueName: \"kubernetes.io/projected/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-kube-api-access-ffxmf\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.759199 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.759157    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-workload-socket\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.759199 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.759184    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-credential-socket\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.759338 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.759207    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-envoy\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.759338 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.759228    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istiod-ca-cert\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.860026 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.859985    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-workload-certs\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.860026 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.860028    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffxmf\" (UniqueName: \"kubernetes.io/projected/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-kube-api-access-ffxmf\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.860322 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.860050    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-workload-socket\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.860322 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.860069    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-credential-socket\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.860322 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.860084    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-envoy\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.860322 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.860114    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istiod-ca-cert\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.860322 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.860144    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-data\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.860322 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.860167    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-token\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.860322 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.860219    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-podinfo\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.860673 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.860491    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-workload-certs\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.860673 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.860528    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-credential-socket\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.860673 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.860554    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-data\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.860673 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.860637    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-workload-socket\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.860914 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.860895    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istiod-ca-cert\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.862837 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.862814    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-envoy\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.862949 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.862902    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-podinfo\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.882639 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.882614    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffxmf\" (UniqueName: \"kubernetes.io/projected/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-kube-api-access-ffxmf\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:31.895386 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.895358    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-token\") pod \"e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") " pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:32.000123 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:31.999997    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:32.160425 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:32.160397    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"]
Jul 28 15:11:32.162369 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:11:32.162331    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ed2ce2c_f93f_425e_865d_bc60cc7cc399.slice/crio-d81424c28de894f1483130415d38680ff3df323b26c803ce5e4198f5a103374d WatchSource:0}: Error finding container d81424c28de894f1483130415d38680ff3df323b26c803ce5e4198f5a103374d: Status 404 returned error can't find the container with id d81424c28de894f1483130415d38680ff3df323b26c803ce5e4198f5a103374d
Jul 28 15:11:32.164385 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:32.164350    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 15:11:32.164504 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:32.164416    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 15:11:32.164504 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:32.164447    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 15:11:32.557168 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:32.557131    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2" event={"ID":"3ed2ce2c-f93f-425e-865d-bc60cc7cc399","Type":"ContainerStarted","Data":"9fb22bcf2e033ea4d78efe280b8c63108c3f7fe6f75113fcb172a3ab5cf07266"}
Jul 28 15:11:32.557387 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:32.557191    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2" event={"ID":"3ed2ce2c-f93f-425e-865d-bc60cc7cc399","Type":"ContainerStarted","Data":"d81424c28de894f1483130415d38680ff3df323b26c803ce5e4198f5a103374d"}
Jul 28 15:11:32.596446 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:32.596388    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2" podStartSLOduration=1.596370689 podStartE2EDuration="1.596370689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:11:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:11:32.59370776 +0000 UTC m=+3162.395249974" watchObservedRunningTime="2026-07-28 15:11:32.596370689 +0000 UTC m=+3162.397912903"
Jul 28 15:11:33.001135 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:33.001083    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:33.006423 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:33.006390    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:33.561712 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:33.561672    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:33.562895 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:33.562865    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:34.925555 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:34.925508    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"]
Jul 28 15:11:34.930058 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:34.930024    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"
Jul 28 15:11:34.932832 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:34.932800    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-becc22-serving-cert\""
Jul 28 15:11:34.941776 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:34.941744    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"]
Jul 28 15:11:35.090176 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:35.090133    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2rp2\" (UniqueName: \"kubernetes.io/projected/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-kube-api-access-j2rp2\") pod \"maas-api-e2e-api-b-becc22-7bdff47798-d6n8b\" (UID: \"b3390ea2-619d-4fd3-b33d-b743f42a7e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"
Jul 28 15:11:35.090371 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:35.090209    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-maas-api-tls\") pod \"maas-api-e2e-api-b-becc22-7bdff47798-d6n8b\" (UID: \"b3390ea2-619d-4fd3-b33d-b743f42a7e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"
Jul 28 15:11:35.090371 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:35.090290    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-openshift-service-ca\") pod \"maas-api-e2e-api-b-becc22-7bdff47798-d6n8b\" (UID: \"b3390ea2-619d-4fd3-b33d-b743f42a7e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"
Jul 28 15:11:35.191646 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:35.191549    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2rp2\" (UniqueName: \"kubernetes.io/projected/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-kube-api-access-j2rp2\") pod \"maas-api-e2e-api-b-becc22-7bdff47798-d6n8b\" (UID: \"b3390ea2-619d-4fd3-b33d-b743f42a7e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"
Jul 28 15:11:35.191646 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:35.191607    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-maas-api-tls\") pod \"maas-api-e2e-api-b-becc22-7bdff47798-d6n8b\" (UID: \"b3390ea2-619d-4fd3-b33d-b743f42a7e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"
Jul 28 15:11:35.191853 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:11:35.191709    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-serving-cert: secret "maas-api-e2e-api-b-becc22-serving-cert" not found
Jul 28 15:11:35.191853 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:35.191737    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-openshift-service-ca\") pod \"maas-api-e2e-api-b-becc22-7bdff47798-d6n8b\" (UID: \"b3390ea2-619d-4fd3-b33d-b743f42a7e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"
Jul 28 15:11:35.191853 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:11:35.191804    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-maas-api-tls podName:b3390ea2-619d-4fd3-b33d-b743f42a7e6d nodeName:}" failed. No retries permitted until 2026-07-28 15:11:35.691780946 +0000 UTC m=+3165.493323155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-maas-api-tls") pod "maas-api-e2e-api-b-becc22-7bdff47798-d6n8b" (UID: "b3390ea2-619d-4fd3-b33d-b743f42a7e6d") : secret "maas-api-e2e-api-b-becc22-serving-cert" not found
Jul 28 15:11:35.192408 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:35.192388    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-openshift-service-ca\") pod \"maas-api-e2e-api-b-becc22-7bdff47798-d6n8b\" (UID: \"b3390ea2-619d-4fd3-b33d-b743f42a7e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"
Jul 28 15:11:35.203395 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:35.203362    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2rp2\" (UniqueName: \"kubernetes.io/projected/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-kube-api-access-j2rp2\") pod \"maas-api-e2e-api-b-becc22-7bdff47798-d6n8b\" (UID: \"b3390ea2-619d-4fd3-b33d-b743f42a7e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"
Jul 28 15:11:35.697985 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:35.697944    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-maas-api-tls\") pod \"maas-api-e2e-api-b-becc22-7bdff47798-d6n8b\" (UID: \"b3390ea2-619d-4fd3-b33d-b743f42a7e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"
Jul 28 15:11:35.702530 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:35.702493    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-maas-api-tls\") pod \"maas-api-e2e-api-b-becc22-7bdff47798-d6n8b\" (UID: \"b3390ea2-619d-4fd3-b33d-b743f42a7e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"
Jul 28 15:11:35.842679 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:35.842633    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"
Jul 28 15:11:35.979921 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:35.979826    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv"]
Jul 28 15:11:35.984906 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:35.984875    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv"
Jul 28 15:11:35.988448 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:35.988419    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-becc22-dockercfg-zxn6t\""
Jul 28 15:11:35.989083 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:35.989061    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-becc22\""
Jul 28 15:11:35.997223 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:35.997195    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv"]
Jul 28 15:11:36.025796 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.025764    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"]
Jul 28 15:11:36.026780 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:11:36.026750    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3390ea2_619d_4fd3_b33d_b743f42a7e6d.slice/crio-4204959b61d8a4a8350c248cf6fb67df7ccffd03d14470baeb6f43e0eb7e6f6e WatchSource:0}: Error finding container 4204959b61d8a4a8350c248cf6fb67df7ccffd03d14470baeb6f43e0eb7e6f6e: Status 404 returned error can't find the container with id 4204959b61d8a4a8350c248cf6fb67df7ccffd03d14470baeb6f43e0eb7e6f6e
Jul 28 15:11:36.102783 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.102741    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6fwm\" (UniqueName: \"kubernetes.io/projected/8b8cc692-0fea-476a-901e-b918f685238e-kube-api-access-h6fwm\") pod \"payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv\" (UID: \"8b8cc692-0fea-476a-901e-b918f685238e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv"
Jul 28 15:11:36.102972 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.102796    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b8cc692-0fea-476a-901e-b918f685238e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv\" (UID: \"8b8cc692-0fea-476a-901e-b918f685238e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv"
Jul 28 15:11:36.203973 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.203932    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6fwm\" (UniqueName: \"kubernetes.io/projected/8b8cc692-0fea-476a-901e-b918f685238e-kube-api-access-h6fwm\") pod \"payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv\" (UID: \"8b8cc692-0fea-476a-901e-b918f685238e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv"
Jul 28 15:11:36.204178 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.203987    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b8cc692-0fea-476a-901e-b918f685238e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv\" (UID: \"8b8cc692-0fea-476a-901e-b918f685238e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv"
Jul 28 15:11:36.204802 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.204780    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b8cc692-0fea-476a-901e-b918f685238e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv\" (UID: \"8b8cc692-0fea-476a-901e-b918f685238e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv"
Jul 28 15:11:36.242410 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.242318    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6fwm\" (UniqueName: \"kubernetes.io/projected/8b8cc692-0fea-476a-901e-b918f685238e-kube-api-access-h6fwm\") pod \"payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv\" (UID: \"8b8cc692-0fea-476a-901e-b918f685238e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv"
Jul 28 15:11:36.243130 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.243086    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h"]
Jul 28 15:11:36.246874 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.246848    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h"
Jul 28 15:11:36.273818 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.273786    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h"]
Jul 28 15:11:36.299805 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.299772    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv"
Jul 28 15:11:36.405895 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.405854    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj9tc\" (UniqueName: \"kubernetes.io/projected/a60f1b6d-b51b-42d0-bc71-4295368bc105-kube-api-access-tj9tc\") pod \"payload-processing-e2e-api-b-becc22-8488566468-q8p6h\" (UID: \"a60f1b6d-b51b-42d0-bc71-4295368bc105\") " pod="openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h"
Jul 28 15:11:36.406113 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.406067    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a60f1b6d-b51b-42d0-bc71-4295368bc105-plugins-config-volume\") pod \"payload-processing-e2e-api-b-becc22-8488566468-q8p6h\" (UID: \"a60f1b6d-b51b-42d0-bc71-4295368bc105\") " pod="openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h"
Jul 28 15:11:36.462110 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.462056    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv"]
Jul 28 15:11:36.466995 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:11:36.466957    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b8cc692_0fea_476a_901e_b918f685238e.slice/crio-0cdf2bd5b1b96268aae069e4642cb9b0600895398392f9bb1fc345e0b1cde167 WatchSource:0}: Error finding container 0cdf2bd5b1b96268aae069e4642cb9b0600895398392f9bb1fc345e0b1cde167: Status 404 returned error can't find the container with id 0cdf2bd5b1b96268aae069e4642cb9b0600895398392f9bb1fc345e0b1cde167
Jul 28 15:11:36.506946 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.506916    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj9tc\" (UniqueName: \"kubernetes.io/projected/a60f1b6d-b51b-42d0-bc71-4295368bc105-kube-api-access-tj9tc\") pod \"payload-processing-e2e-api-b-becc22-8488566468-q8p6h\" (UID: \"a60f1b6d-b51b-42d0-bc71-4295368bc105\") " pod="openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h"
Jul 28 15:11:36.507131 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.507031    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a60f1b6d-b51b-42d0-bc71-4295368bc105-plugins-config-volume\") pod \"payload-processing-e2e-api-b-becc22-8488566468-q8p6h\" (UID: \"a60f1b6d-b51b-42d0-bc71-4295368bc105\") " pod="openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h"
Jul 28 15:11:36.507952 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.507927    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a60f1b6d-b51b-42d0-bc71-4295368bc105-plugins-config-volume\") pod \"payload-processing-e2e-api-b-becc22-8488566468-q8p6h\" (UID: \"a60f1b6d-b51b-42d0-bc71-4295368bc105\") " pod="openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h"
Jul 28 15:11:36.519178 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.519139    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj9tc\" (UniqueName: \"kubernetes.io/projected/a60f1b6d-b51b-42d0-bc71-4295368bc105-kube-api-access-tj9tc\") pod \"payload-processing-e2e-api-b-becc22-8488566468-q8p6h\" (UID: \"a60f1b6d-b51b-42d0-bc71-4295368bc105\") " pod="openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h"
Jul 28 15:11:36.560380 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.560339    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h"
Jul 28 15:11:36.580487 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.580440    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv" event={"ID":"8b8cc692-0fea-476a-901e-b918f685238e","Type":"ContainerStarted","Data":"bd279ccca358195f5d767e445f30fee971fac0235ddf7ccb60f28fba7493cd2d"}
Jul 28 15:11:36.580682 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.580634    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv"
Jul 28 15:11:36.580682 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.580667    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv" event={"ID":"8b8cc692-0fea-476a-901e-b918f685238e","Type":"ContainerStarted","Data":"0cdf2bd5b1b96268aae069e4642cb9b0600895398392f9bb1fc345e0b1cde167"}
Jul 28 15:11:36.582201 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.582176    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b" event={"ID":"b3390ea2-619d-4fd3-b33d-b743f42a7e6d","Type":"ContainerStarted","Data":"315c092d1f8c854a02eaa186727a5f7b4b1b45c4bb26788fd5df593554517d73"}
Jul 28 15:11:36.582335 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.582215    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b" event={"ID":"b3390ea2-619d-4fd3-b33d-b743f42a7e6d","Type":"ContainerStarted","Data":"4204959b61d8a4a8350c248cf6fb67df7ccffd03d14470baeb6f43e0eb7e6f6e"}
Jul 28 15:11:36.582335 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.582318    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"
Jul 28 15:11:36.622998 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.622946    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv" podStartSLOduration=1.622927297 podStartE2EDuration="1.622927297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:11:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:11:36.621520448 +0000 UTC m=+3166.423062665" watchObservedRunningTime="2026-07-28 15:11:36.622927297 +0000 UTC m=+3166.424469532"
Jul 28 15:11:36.686122 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.686036    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b" podStartSLOduration=2.247244421 podStartE2EDuration="2.686012019s" totalImagesPullingTime="438.767598ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:11:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:11:36.682327222 +0000 UTC m=+3166.483869452" watchObservedRunningTime="2026-07-28 15:11:36.686012019 +0000 UTC m=+3166.487554234"
Jul 28 15:11:36.737755 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:36.737724    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h"]
Jul 28 15:11:36.738647 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:11:36.738604    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda60f1b6d_b51b_42d0_bc71_4295368bc105.slice/crio-a0698fa309b57fe728bcd31190223fc6ca6c2056759a5c4a733c6554cc91b309 WatchSource:0}: Error finding container a0698fa309b57fe728bcd31190223fc6ca6c2056759a5c4a733c6554cc91b309: Status 404 returned error can't find the container with id a0698fa309b57fe728bcd31190223fc6ca6c2056759a5c4a733c6554cc91b309
Jul 28 15:11:37.591524 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:37.591477    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h" event={"ID":"a60f1b6d-b51b-42d0-bc71-4295368bc105","Type":"ContainerStarted","Data":"a53e1cfc3a452bd8fab5008d72144284865523bed73bb157eca7ea123530ae8b"}
Jul 28 15:11:37.591943 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:37.591560    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h" event={"ID":"a60f1b6d-b51b-42d0-bc71-4295368bc105","Type":"ContainerStarted","Data":"a0698fa309b57fe728bcd31190223fc6ca6c2056759a5c4a733c6554cc91b309"}
Jul 28 15:11:37.640285 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:37.640225    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h" podStartSLOduration=1.6402046700000001 podStartE2EDuration="1.64020467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:11:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:11:37.639278316 +0000 UTC m=+3167.440820532" watchObservedRunningTime="2026-07-28 15:11:37.64020467 +0000 UTC m=+3167.441746879"
Jul 28 15:11:38.596775 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:38.596738    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h"
Jul 28 15:11:42.597634 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:42.597605    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"
Jul 28 15:11:47.594223 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:47.594187    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv"
Jul 28 15:11:49.604290 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:49.604244    2584 patch_prober.go:28] interesting pod/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 15:11:49.604717 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:49.604295    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h"
Jul 28 15:11:49.604717 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:49.604327    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2" podUID="3ed2ce2c-f93f-425e-865d-bc60cc7cc399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 15:11:51.672655 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:51.672558    2584 generic.go:356] "Generic (PLEG): container finished" podID="3ed2ce2c-f93f-425e-865d-bc60cc7cc399" containerID="9fb22bcf2e033ea4d78efe280b8c63108c3f7fe6f75113fcb172a3ab5cf07266" exitCode=0
Jul 28 15:11:51.672655 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:51.672630    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2" event={"ID":"3ed2ce2c-f93f-425e-865d-bc60cc7cc399","Type":"ContainerDied","Data":"9fb22bcf2e033ea4d78efe280b8c63108c3f7fe6f75113fcb172a3ab5cf07266"}
Jul 28 15:11:51.673149 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:51.673057    2584 scope.go:117] "RemoveContainer" containerID="9fb22bcf2e033ea4d78efe280b8c63108c3f7fe6f75113fcb172a3ab5cf07266"
Jul 28 15:11:51.674133 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:51.674086    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 15:11:51.674239 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:51.674189    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 15:11:51.674239 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:51.674218    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 15:11:52.680111 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:52.680063    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2" event={"ID":"3ed2ce2c-f93f-425e-865d-bc60cc7cc399","Type":"ContainerStarted","Data":"aada01bde3d564aeb5e6a5a517ff4a3da82850ec4727bc36ebaa3d2156abc50b"}
Jul 28 15:11:53.000807 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:53.000708    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:53.001173 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:53.001147    2584 patch_prober.go:28] interesting pod/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Jul 28 15:11:53.001285 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:53.001207    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2" podUID="3ed2ce2c-f93f-425e-865d-bc60cc7cc399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Jul 28 15:11:54.001058 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:54.001015    2584 patch_prober.go:28] interesting pod/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Jul 28 15:11:54.001492 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:54.001075    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2" podUID="3ed2ce2c-f93f-425e-865d-bc60cc7cc399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Jul 28 15:11:55.001394 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:55.001356    2584 patch_prober.go:28] interesting pod/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Jul 28 15:11:55.001887 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:55.001426    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2" podUID="3ed2ce2c-f93f-425e-865d-bc60cc7cc399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Jul 28 15:11:56.000935 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:56.000897    2584 patch_prober.go:28] interesting pod/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Jul 28 15:11:56.001135 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:56.000956    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2" podUID="3ed2ce2c-f93f-425e-865d-bc60cc7cc399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Jul 28 15:11:57.004825 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:57.004792    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:57.005245 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:57.005223    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:11:57.006003 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:11:57.005984    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:12:18.612981 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:18.612934    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"]
Jul 28 15:12:18.613654 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:18.613246    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7" podUID="d078f02c-9add-4f94-80a6-01830b0e4d5f" containerName="maas-api" containerID="cri-o://50b6c7bfc877fb9ae1f202aed95464e1fc6ad7c173bbd8e876db981a47ee52e5" gracePeriod=30
Jul 28 15:12:18.684419 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:18.684374    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v"]
Jul 28 15:12:18.684678 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:18.684613    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v" podUID="1a135e30-9600-4086-9000-79683e9e4dad" containerName="payload-processing" containerID="cri-o://28babe247be6fe4ec50cc29f9ba426d7ca88f7259c5df19ed09ca66897258a61" gracePeriod=30
Jul 28 15:12:18.701840 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:18.701805    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h"]
Jul 28 15:12:18.702134 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:18.702063    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h" podUID="a7ba30f1-a2c2-4484-b6f3-4873b304876b" containerName="payload-pre-processing" containerID="cri-o://040905e3991c23335a2599bc53b5b60b75a04fbf7b77cc72fc53ab253099a418" gracePeriod=30
Jul 28 15:12:18.811533 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:18.811498    2584 generic.go:356] "Generic (PLEG): container finished" podID="d078f02c-9add-4f94-80a6-01830b0e4d5f" containerID="50b6c7bfc877fb9ae1f202aed95464e1fc6ad7c173bbd8e876db981a47ee52e5" exitCode=0
Jul 28 15:12:18.811671 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:18.811578    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7" event={"ID":"d078f02c-9add-4f94-80a6-01830b0e4d5f","Type":"ContainerDied","Data":"50b6c7bfc877fb9ae1f202aed95464e1fc6ad7c173bbd8e876db981a47ee52e5"}
Jul 28 15:12:18.813552 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:18.813520    2584 generic.go:356] "Generic (PLEG): container finished" podID="1a135e30-9600-4086-9000-79683e9e4dad" containerID="28babe247be6fe4ec50cc29f9ba426d7ca88f7259c5df19ed09ca66897258a61" exitCode=0
Jul 28 15:12:18.813715 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:18.813594    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v" event={"ID":"1a135e30-9600-4086-9000-79683e9e4dad","Type":"ContainerDied","Data":"28babe247be6fe4ec50cc29f9ba426d7ca88f7259c5df19ed09ca66897258a61"}
Jul 28 15:12:18.994428 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:18.994397    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v"
Jul 28 15:12:19.020991 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.020963    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"
Jul 28 15:12:19.032496 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.032467    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h"
Jul 28 15:12:19.132577 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.132472    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d078f02c-9add-4f94-80a6-01830b0e4d5f-openshift-service-ca\") pod \"d078f02c-9add-4f94-80a6-01830b0e4d5f\" (UID: \"d078f02c-9add-4f94-80a6-01830b0e4d5f\") "
Jul 28 15:12:19.132577 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.132536    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7ba30f1-a2c2-4484-b6f3-4873b304876b-plugins-config-volume\") pod \"a7ba30f1-a2c2-4484-b6f3-4873b304876b\" (UID: \"a7ba30f1-a2c2-4484-b6f3-4873b304876b\") "
Jul 28 15:12:19.132577 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.132563    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d078f02c-9add-4f94-80a6-01830b0e4d5f-maas-api-tls\") pod \"d078f02c-9add-4f94-80a6-01830b0e4d5f\" (UID: \"d078f02c-9add-4f94-80a6-01830b0e4d5f\") "
Jul 28 15:12:19.132883 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.132604    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfcgx\" (UniqueName: \"kubernetes.io/projected/1a135e30-9600-4086-9000-79683e9e4dad-kube-api-access-dfcgx\") pod \"1a135e30-9600-4086-9000-79683e9e4dad\" (UID: \"1a135e30-9600-4086-9000-79683e9e4dad\") "
Jul 28 15:12:19.132883 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.132647    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a135e30-9600-4086-9000-79683e9e4dad-plugins-config-volume\") pod \"1a135e30-9600-4086-9000-79683e9e4dad\" (UID: \"1a135e30-9600-4086-9000-79683e9e4dad\") "
Jul 28 15:12:19.132883 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.132702    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqc75\" (UniqueName: \"kubernetes.io/projected/d078f02c-9add-4f94-80a6-01830b0e4d5f-kube-api-access-nqc75\") pod \"d078f02c-9add-4f94-80a6-01830b0e4d5f\" (UID: \"d078f02c-9add-4f94-80a6-01830b0e4d5f\") "
Jul 28 15:12:19.132883 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.132749    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fszd\" (UniqueName: \"kubernetes.io/projected/a7ba30f1-a2c2-4484-b6f3-4873b304876b-kube-api-access-6fszd\") pod \"a7ba30f1-a2c2-4484-b6f3-4873b304876b\" (UID: \"a7ba30f1-a2c2-4484-b6f3-4873b304876b\") "
Jul 28 15:12:19.133129 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.132974    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d078f02c-9add-4f94-80a6-01830b0e4d5f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d078f02c-9add-4f94-80a6-01830b0e4d5f" (UID: "d078f02c-9add-4f94-80a6-01830b0e4d5f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:12:19.133129 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.133016    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7ba30f1-a2c2-4484-b6f3-4873b304876b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a7ba30f1-a2c2-4484-b6f3-4873b304876b" (UID: "a7ba30f1-a2c2-4484-b6f3-4873b304876b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:12:19.133129 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.133048    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a135e30-9600-4086-9000-79683e9e4dad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1a135e30-9600-4086-9000-79683e9e4dad" (UID: "1a135e30-9600-4086-9000-79683e9e4dad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:12:19.133266 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.133181    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d078f02c-9add-4f94-80a6-01830b0e4d5f-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:19.133266 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.133199    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7ba30f1-a2c2-4484-b6f3-4873b304876b-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:19.133266 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.133213    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a135e30-9600-4086-9000-79683e9e4dad-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:19.134778 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.134758    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d078f02c-9add-4f94-80a6-01830b0e4d5f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d078f02c-9add-4f94-80a6-01830b0e4d5f" (UID: "d078f02c-9add-4f94-80a6-01830b0e4d5f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:12:19.134917 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.134898    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7ba30f1-a2c2-4484-b6f3-4873b304876b-kube-api-access-6fszd" (OuterVolumeSpecName: "kube-api-access-6fszd") pod "a7ba30f1-a2c2-4484-b6f3-4873b304876b" (UID: "a7ba30f1-a2c2-4484-b6f3-4873b304876b"). InnerVolumeSpecName "kube-api-access-6fszd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:12:19.135224 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.135206    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d078f02c-9add-4f94-80a6-01830b0e4d5f-kube-api-access-nqc75" (OuterVolumeSpecName: "kube-api-access-nqc75") pod "d078f02c-9add-4f94-80a6-01830b0e4d5f" (UID: "d078f02c-9add-4f94-80a6-01830b0e4d5f"). InnerVolumeSpecName "kube-api-access-nqc75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:12:19.135277 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.135245    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a135e30-9600-4086-9000-79683e9e4dad-kube-api-access-dfcgx" (OuterVolumeSpecName: "kube-api-access-dfcgx") pod "1a135e30-9600-4086-9000-79683e9e4dad" (UID: "1a135e30-9600-4086-9000-79683e9e4dad"). InnerVolumeSpecName "kube-api-access-dfcgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:12:19.234650 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.234595    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqc75\" (UniqueName: \"kubernetes.io/projected/d078f02c-9add-4f94-80a6-01830b0e4d5f-kube-api-access-nqc75\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:19.234650 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.234643    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fszd\" (UniqueName: \"kubernetes.io/projected/a7ba30f1-a2c2-4484-b6f3-4873b304876b-kube-api-access-6fszd\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:19.234650 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.234658    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d078f02c-9add-4f94-80a6-01830b0e4d5f-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:19.234928 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.234674    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfcgx\" (UniqueName: \"kubernetes.io/projected/1a135e30-9600-4086-9000-79683e9e4dad-kube-api-access-dfcgx\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:19.820509 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.820482    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"
Jul 28 15:12:19.820969 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.820504    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7" event={"ID":"d078f02c-9add-4f94-80a6-01830b0e4d5f","Type":"ContainerDied","Data":"6ff26b64e402b1d179618696251305f975c1391cb8f09108a39403ee9ce01809"}
Jul 28 15:12:19.820969 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.820577    2584 scope.go:117] "RemoveContainer" containerID="50b6c7bfc877fb9ae1f202aed95464e1fc6ad7c173bbd8e876db981a47ee52e5"
Jul 28 15:12:19.821882 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.821852    2584 generic.go:356] "Generic (PLEG): container finished" podID="a7ba30f1-a2c2-4484-b6f3-4873b304876b" containerID="040905e3991c23335a2599bc53b5b60b75a04fbf7b77cc72fc53ab253099a418" exitCode=0
Jul 28 15:12:19.821945 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.821898    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h"
Jul 28 15:12:19.821945 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.821923    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h" event={"ID":"a7ba30f1-a2c2-4484-b6f3-4873b304876b","Type":"ContainerDied","Data":"040905e3991c23335a2599bc53b5b60b75a04fbf7b77cc72fc53ab253099a418"}
Jul 28 15:12:19.822039 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.821963    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h" event={"ID":"a7ba30f1-a2c2-4484-b6f3-4873b304876b","Type":"ContainerDied","Data":"911ad10f99ad466704869cfd83a9ee0cff68991c74dc93c4fc8940064d032319"}
Jul 28 15:12:19.823410 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.823381    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v" event={"ID":"1a135e30-9600-4086-9000-79683e9e4dad","Type":"ContainerDied","Data":"2ea68ed56ce29d2234b7f4491f3bb06c62a636ffb3c6d5e5ebe4bd339dc1586e"}
Jul 28 15:12:19.823526 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.823384    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v"
Jul 28 15:12:19.832388 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.832371    2584 scope.go:117] "RemoveContainer" containerID="040905e3991c23335a2599bc53b5b60b75a04fbf7b77cc72fc53ab253099a418"
Jul 28 15:12:19.843171 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.843128    2584 scope.go:117] "RemoveContainer" containerID="040905e3991c23335a2599bc53b5b60b75a04fbf7b77cc72fc53ab253099a418"
Jul 28 15:12:19.843488 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:12:19.843466    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"040905e3991c23335a2599bc53b5b60b75a04fbf7b77cc72fc53ab253099a418\": container with ID starting with 040905e3991c23335a2599bc53b5b60b75a04fbf7b77cc72fc53ab253099a418 not found: ID does not exist" containerID="040905e3991c23335a2599bc53b5b60b75a04fbf7b77cc72fc53ab253099a418"
Jul 28 15:12:19.843567 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.843504    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"040905e3991c23335a2599bc53b5b60b75a04fbf7b77cc72fc53ab253099a418"} err="failed to get container status \"040905e3991c23335a2599bc53b5b60b75a04fbf7b77cc72fc53ab253099a418\": rpc error: code = NotFound desc = could not find container \"040905e3991c23335a2599bc53b5b60b75a04fbf7b77cc72fc53ab253099a418\": container with ID starting with 040905e3991c23335a2599bc53b5b60b75a04fbf7b77cc72fc53ab253099a418 not found: ID does not exist"
Jul 28 15:12:19.843567 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.843529    2584 scope.go:117] "RemoveContainer" containerID="28babe247be6fe4ec50cc29f9ba426d7ca88f7259c5df19ed09ca66897258a61"
Jul 28 15:12:19.851390 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.851357    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"]
Jul 28 15:12:19.859471 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.859436    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cef5a5-bd7cb8fd9-jn7p7"]
Jul 28 15:12:19.884200 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.882968    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v"]
Jul 28 15:12:19.889616 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.889579    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cef5a5-65748cf47d-whw5v"]
Jul 28 15:12:19.911374 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.911338    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h"]
Jul 28 15:12:19.929172 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:19.929134    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cef5a5-5b976c96dd-pvp2h"]
Jul 28 15:12:20.797168 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:20.797129    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a135e30-9600-4086-9000-79683e9e4dad" path="/var/lib/kubelet/pods/1a135e30-9600-4086-9000-79683e9e4dad/volumes"
Jul 28 15:12:20.797544 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:20.797530    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7ba30f1-a2c2-4484-b6f3-4873b304876b" path="/var/lib/kubelet/pods/a7ba30f1-a2c2-4484-b6f3-4873b304876b/volumes"
Jul 28 15:12:20.797881 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:20.797869    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d078f02c-9add-4f94-80a6-01830b0e4d5f" path="/var/lib/kubelet/pods/d078f02c-9add-4f94-80a6-01830b0e4d5f/volumes"
Jul 28 15:12:35.647398 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.647353    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"]
Jul 28 15:12:35.647917 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.647665    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b" podUID="b3390ea2-619d-4fd3-b33d-b743f42a7e6d" containerName="maas-api" containerID="cri-o://315c092d1f8c854a02eaa186727a5f7b4b1b45c4bb26788fd5df593554517d73" gracePeriod=30
Jul 28 15:12:35.774919 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.774887    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h"]
Jul 28 15:12:35.775196 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.775147    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h" podUID="a60f1b6d-b51b-42d0-bc71-4295368bc105" containerName="payload-processing" containerID="cri-o://a53e1cfc3a452bd8fab5008d72144284865523bed73bb157eca7ea123530ae8b" gracePeriod=30
Jul 28 15:12:35.785437 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.785384    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv"]
Jul 28 15:12:35.785666 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.785639    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv" podUID="8b8cc692-0fea-476a-901e-b918f685238e" containerName="payload-pre-processing" containerID="cri-o://bd279ccca358195f5d767e445f30fee971fac0235ddf7ccb60f28fba7493cd2d" gracePeriod=30
Jul 28 15:12:35.909051 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.909019    2584 generic.go:356] "Generic (PLEG): container finished" podID="a60f1b6d-b51b-42d0-bc71-4295368bc105" containerID="a53e1cfc3a452bd8fab5008d72144284865523bed73bb157eca7ea123530ae8b" exitCode=0
Jul 28 15:12:35.909234 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.909114    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h" event={"ID":"a60f1b6d-b51b-42d0-bc71-4295368bc105","Type":"ContainerDied","Data":"a53e1cfc3a452bd8fab5008d72144284865523bed73bb157eca7ea123530ae8b"}
Jul 28 15:12:35.910585 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.910549    2584 generic.go:356] "Generic (PLEG): container finished" podID="b3390ea2-619d-4fd3-b33d-b743f42a7e6d" containerID="315c092d1f8c854a02eaa186727a5f7b4b1b45c4bb26788fd5df593554517d73" exitCode=0
Jul 28 15:12:35.910768 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.910745    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b" event={"ID":"b3390ea2-619d-4fd3-b33d-b743f42a7e6d","Type":"ContainerDied","Data":"315c092d1f8c854a02eaa186727a5f7b4b1b45c4bb26788fd5df593554517d73"}
Jul 28 15:12:35.910869 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.910799    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b" event={"ID":"b3390ea2-619d-4fd3-b33d-b743f42a7e6d","Type":"ContainerDied","Data":"4204959b61d8a4a8350c248cf6fb67df7ccffd03d14470baeb6f43e0eb7e6f6e"}
Jul 28 15:12:35.910869 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.910826    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4204959b61d8a4a8350c248cf6fb67df7ccffd03d14470baeb6f43e0eb7e6f6e"
Jul 28 15:12:35.913005 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.912979    2584 generic.go:356] "Generic (PLEG): container finished" podID="8b8cc692-0fea-476a-901e-b918f685238e" containerID="bd279ccca358195f5d767e445f30fee971fac0235ddf7ccb60f28fba7493cd2d" exitCode=0
Jul 28 15:12:35.913125 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.913049    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv" event={"ID":"8b8cc692-0fea-476a-901e-b918f685238e","Type":"ContainerDied","Data":"bd279ccca358195f5d767e445f30fee971fac0235ddf7ccb60f28fba7493cd2d"}
Jul 28 15:12:35.919379 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.919355    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"
Jul 28 15:12:35.980456 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.980429    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-openshift-service-ca\") pod \"b3390ea2-619d-4fd3-b33d-b743f42a7e6d\" (UID: \"b3390ea2-619d-4fd3-b33d-b743f42a7e6d\") "
Jul 28 15:12:35.980456 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.980468    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2rp2\" (UniqueName: \"kubernetes.io/projected/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-kube-api-access-j2rp2\") pod \"b3390ea2-619d-4fd3-b33d-b743f42a7e6d\" (UID: \"b3390ea2-619d-4fd3-b33d-b743f42a7e6d\") "
Jul 28 15:12:35.980704 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.980541    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-maas-api-tls\") pod \"b3390ea2-619d-4fd3-b33d-b743f42a7e6d\" (UID: \"b3390ea2-619d-4fd3-b33d-b743f42a7e6d\") "
Jul 28 15:12:35.980967 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.980909    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b3390ea2-619d-4fd3-b33d-b743f42a7e6d" (UID: "b3390ea2-619d-4fd3-b33d-b743f42a7e6d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:12:35.982818 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.982791    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b3390ea2-619d-4fd3-b33d-b743f42a7e6d" (UID: "b3390ea2-619d-4fd3-b33d-b743f42a7e6d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:12:35.983149 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:35.983128    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-kube-api-access-j2rp2" (OuterVolumeSpecName: "kube-api-access-j2rp2") pod "b3390ea2-619d-4fd3-b33d-b743f42a7e6d" (UID: "b3390ea2-619d-4fd3-b33d-b743f42a7e6d"). InnerVolumeSpecName "kube-api-access-j2rp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:12:36.058185 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.058161    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h"
Jul 28 15:12:36.063648 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.063627    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv"
Jul 28 15:12:36.081281 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.081251    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a60f1b6d-b51b-42d0-bc71-4295368bc105-plugins-config-volume\") pod \"a60f1b6d-b51b-42d0-bc71-4295368bc105\" (UID: \"a60f1b6d-b51b-42d0-bc71-4295368bc105\") "
Jul 28 15:12:36.081461 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.081313    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tj9tc\" (UniqueName: \"kubernetes.io/projected/a60f1b6d-b51b-42d0-bc71-4295368bc105-kube-api-access-tj9tc\") pod \"a60f1b6d-b51b-42d0-bc71-4295368bc105\" (UID: \"a60f1b6d-b51b-42d0-bc71-4295368bc105\") "
Jul 28 15:12:36.081534 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.081520    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:36.081534 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.081530    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2rp2\" (UniqueName: \"kubernetes.io/projected/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-kube-api-access-j2rp2\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:36.081643 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.081539    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3390ea2-619d-4fd3-b33d-b743f42a7e6d-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:36.081643 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.081565    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a60f1b6d-b51b-42d0-bc71-4295368bc105-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a60f1b6d-b51b-42d0-bc71-4295368bc105" (UID: "a60f1b6d-b51b-42d0-bc71-4295368bc105"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:12:36.084565 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.084537    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a60f1b6d-b51b-42d0-bc71-4295368bc105-kube-api-access-tj9tc" (OuterVolumeSpecName: "kube-api-access-tj9tc") pod "a60f1b6d-b51b-42d0-bc71-4295368bc105" (UID: "a60f1b6d-b51b-42d0-bc71-4295368bc105"). InnerVolumeSpecName "kube-api-access-tj9tc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:12:36.182238 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.182153    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b8cc692-0fea-476a-901e-b918f685238e-plugins-config-volume\") pod \"8b8cc692-0fea-476a-901e-b918f685238e\" (UID: \"8b8cc692-0fea-476a-901e-b918f685238e\") "
Jul 28 15:12:36.182389 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.182254    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6fwm\" (UniqueName: \"kubernetes.io/projected/8b8cc692-0fea-476a-901e-b918f685238e-kube-api-access-h6fwm\") pod \"8b8cc692-0fea-476a-901e-b918f685238e\" (UID: \"8b8cc692-0fea-476a-901e-b918f685238e\") "
Jul 28 15:12:36.182469 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.182457    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a60f1b6d-b51b-42d0-bc71-4295368bc105-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:36.182509 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.182471    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tj9tc\" (UniqueName: \"kubernetes.io/projected/a60f1b6d-b51b-42d0-bc71-4295368bc105-kube-api-access-tj9tc\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:36.182612 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.182586    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b8cc692-0fea-476a-901e-b918f685238e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8b8cc692-0fea-476a-901e-b918f685238e" (UID: "8b8cc692-0fea-476a-901e-b918f685238e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:12:36.184274 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.184249    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b8cc692-0fea-476a-901e-b918f685238e-kube-api-access-h6fwm" (OuterVolumeSpecName: "kube-api-access-h6fwm") pod "8b8cc692-0fea-476a-901e-b918f685238e" (UID: "8b8cc692-0fea-476a-901e-b918f685238e"). InnerVolumeSpecName "kube-api-access-h6fwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:12:36.283627 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.283598    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b8cc692-0fea-476a-901e-b918f685238e-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:36.283627 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.283624    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6fwm\" (UniqueName: \"kubernetes.io/projected/8b8cc692-0fea-476a-901e-b918f685238e-kube-api-access-h6fwm\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:36.919486 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.919398    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv"
Jul 28 15:12:36.919486 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.919452    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv" event={"ID":"8b8cc692-0fea-476a-901e-b918f685238e","Type":"ContainerDied","Data":"0cdf2bd5b1b96268aae069e4642cb9b0600895398392f9bb1fc345e0b1cde167"}
Jul 28 15:12:36.919998 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.919521    2584 scope.go:117] "RemoveContainer" containerID="bd279ccca358195f5d767e445f30fee971fac0235ddf7ccb60f28fba7493cd2d"
Jul 28 15:12:36.920866 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.920849    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h"
Jul 28 15:12:36.920931 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.920862    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"
Jul 28 15:12:36.920931 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.920892    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h" event={"ID":"a60f1b6d-b51b-42d0-bc71-4295368bc105","Type":"ContainerDied","Data":"a0698fa309b57fe728bcd31190223fc6ca6c2056759a5c4a733c6554cc91b309"}
Jul 28 15:12:36.929135 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.929110    2584 scope.go:117] "RemoveContainer" containerID="a53e1cfc3a452bd8fab5008d72144284865523bed73bb157eca7ea123530ae8b"
Jul 28 15:12:36.950652 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.950629    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h"]
Jul 28 15:12:36.957909 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.957886    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-becc22-8488566468-q8p6h"]
Jul 28 15:12:36.972257 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.972230    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"]
Jul 28 15:12:36.982902 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.982877    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-becc22-7bdff47798-d6n8b"]
Jul 28 15:12:36.996204 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:36.996163    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv"]
Jul 28 15:12:37.001211 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:37.001192    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-becc22-5fd69cb75b-xk7fv"]
Jul 28 15:12:38.798791 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:38.798756    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b8cc692-0fea-476a-901e-b918f685238e" path="/var/lib/kubelet/pods/8b8cc692-0fea-476a-901e-b918f685238e/volumes"
Jul 28 15:12:38.799177 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:38.799159    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a60f1b6d-b51b-42d0-bc71-4295368bc105" path="/var/lib/kubelet/pods/a60f1b6d-b51b-42d0-bc71-4295368bc105/volumes"
Jul 28 15:12:38.799916 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:38.799572    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3390ea2-619d-4fd3-b33d-b743f42a7e6d" path="/var/lib/kubelet/pods/b3390ea2-619d-4fd3-b33d-b743f42a7e6d/volumes"
Jul 28 15:12:44.216486 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:44.216406    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"]
Jul 28 15:12:44.216903 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:44.216680    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2" podUID="3ed2ce2c-f93f-425e-865d-bc60cc7cc399" containerName="istio-proxy" containerID="cri-o://aada01bde3d564aeb5e6a5a517ff4a3da82850ec4727bc36ebaa3d2156abc50b" gracePeriod=30
Jul 28 15:12:48.582363 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.582316    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"]
Jul 28 15:12:48.582797 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.582779    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3390ea2-619d-4fd3-b33d-b743f42a7e6d" containerName="maas-api"
Jul 28 15:12:48.582847 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.582799    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3390ea2-619d-4fd3-b33d-b743f42a7e6d" containerName="maas-api"
Jul 28 15:12:48.582847 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.582809    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d078f02c-9add-4f94-80a6-01830b0e4d5f" containerName="maas-api"
Jul 28 15:12:48.582847 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.582814    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d078f02c-9add-4f94-80a6-01830b0e4d5f" containerName="maas-api"
Jul 28 15:12:48.582847 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.582825    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7ba30f1-a2c2-4484-b6f3-4873b304876b" containerName="payload-pre-processing"
Jul 28 15:12:48.582847 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.582831    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7ba30f1-a2c2-4484-b6f3-4873b304876b" containerName="payload-pre-processing"
Jul 28 15:12:48.582847 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.582837    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b8cc692-0fea-476a-901e-b918f685238e" containerName="payload-pre-processing"
Jul 28 15:12:48.582847 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.582842    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b8cc692-0fea-476a-901e-b918f685238e" containerName="payload-pre-processing"
Jul 28 15:12:48.583058 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.582855    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a135e30-9600-4086-9000-79683e9e4dad" containerName="payload-processing"
Jul 28 15:12:48.583058 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.582860    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a135e30-9600-4086-9000-79683e9e4dad" containerName="payload-processing"
Jul 28 15:12:48.583058 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.582874    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a60f1b6d-b51b-42d0-bc71-4295368bc105" containerName="payload-processing"
Jul 28 15:12:48.583058 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.582879    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a60f1b6d-b51b-42d0-bc71-4295368bc105" containerName="payload-processing"
Jul 28 15:12:48.583058 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.582933    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a135e30-9600-4086-9000-79683e9e4dad" containerName="payload-processing"
Jul 28 15:12:48.583058 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.582941    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d078f02c-9add-4f94-80a6-01830b0e4d5f" containerName="maas-api"
Jul 28 15:12:48.583058 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.582949    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a60f1b6d-b51b-42d0-bc71-4295368bc105" containerName="payload-processing"
Jul 28 15:12:48.583058 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.582956    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3390ea2-619d-4fd3-b33d-b743f42a7e6d" containerName="maas-api"
Jul 28 15:12:48.583058 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.582964    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7ba30f1-a2c2-4484-b6f3-4873b304876b" containerName="payload-pre-processing"
Jul 28 15:12:48.583058 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.582972    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b8cc692-0fea-476a-901e-b918f685238e" containerName="payload-pre-processing"
Jul 28 15:12:48.589939 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.586411    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"
Jul 28 15:12:48.597493 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.597468    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-d96a19-serving-cert\""
Jul 28 15:12:48.627940 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.627908    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"]
Jul 28 15:12:48.697644 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.697608    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33658123-a2fc-4f67-bb97-03711502555c-maas-api-tls\") pod \"maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc\" (UID: \"33658123-a2fc-4f67-bb97-03711502555c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"
Jul 28 15:12:48.697820 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.697705    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxjrc\" (UniqueName: \"kubernetes.io/projected/33658123-a2fc-4f67-bb97-03711502555c-kube-api-access-kxjrc\") pod \"maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc\" (UID: \"33658123-a2fc-4f67-bb97-03711502555c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"
Jul 28 15:12:48.697820 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.697732    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33658123-a2fc-4f67-bb97-03711502555c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc\" (UID: \"33658123-a2fc-4f67-bb97-03711502555c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"
Jul 28 15:12:48.798837 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.798804    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33658123-a2fc-4f67-bb97-03711502555c-maas-api-tls\") pod \"maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc\" (UID: \"33658123-a2fc-4f67-bb97-03711502555c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"
Jul 28 15:12:48.799023 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.798892    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxjrc\" (UniqueName: \"kubernetes.io/projected/33658123-a2fc-4f67-bb97-03711502555c-kube-api-access-kxjrc\") pod \"maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc\" (UID: \"33658123-a2fc-4f67-bb97-03711502555c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"
Jul 28 15:12:48.799023 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.798912    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33658123-a2fc-4f67-bb97-03711502555c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc\" (UID: \"33658123-a2fc-4f67-bb97-03711502555c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"
Jul 28 15:12:48.799023 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:12:48.798944    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-serving-cert: secret "maas-api-e2e-shared-a-d96a19-serving-cert" not found
Jul 28 15:12:48.799023 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:12:48.799012    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33658123-a2fc-4f67-bb97-03711502555c-maas-api-tls podName:33658123-a2fc-4f67-bb97-03711502555c nodeName:}" failed. No retries permitted until 2026-07-28 15:12:49.29899513 +0000 UTC m=+3239.100537322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/33658123-a2fc-4f67-bb97-03711502555c-maas-api-tls") pod "maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc" (UID: "33658123-a2fc-4f67-bb97-03711502555c") : secret "maas-api-e2e-shared-a-d96a19-serving-cert" not found
Jul 28 15:12:48.799606 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.799586    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33658123-a2fc-4f67-bb97-03711502555c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc\" (UID: \"33658123-a2fc-4f67-bb97-03711502555c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"
Jul 28 15:12:48.808488 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:48.808463    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxjrc\" (UniqueName: \"kubernetes.io/projected/33658123-a2fc-4f67-bb97-03711502555c-kube-api-access-kxjrc\") pod \"maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc\" (UID: \"33658123-a2fc-4f67-bb97-03711502555c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"
Jul 28 15:12:49.305390 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.305343    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33658123-a2fc-4f67-bb97-03711502555c-maas-api-tls\") pod \"maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc\" (UID: \"33658123-a2fc-4f67-bb97-03711502555c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"
Jul 28 15:12:49.307766 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.307743    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33658123-a2fc-4f67-bb97-03711502555c-maas-api-tls\") pod \"maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc\" (UID: \"33658123-a2fc-4f67-bb97-03711502555c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"
Jul 28 15:12:49.478380 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.478315    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:12:49.481373 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.481346    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j"]
Jul 28 15:12:49.481754 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.481739    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ed2ce2c-f93f-425e-865d-bc60cc7cc399" containerName="istio-proxy"
Jul 28 15:12:49.481754 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.481755    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ed2ce2c-f93f-425e-865d-bc60cc7cc399" containerName="istio-proxy"
Jul 28 15:12:49.481882 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.481767    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ed2ce2c-f93f-425e-865d-bc60cc7cc399" containerName="istio-proxy"
Jul 28 15:12:49.481882 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.481773    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ed2ce2c-f93f-425e-865d-bc60cc7cc399" containerName="istio-proxy"
Jul 28 15:12:49.481882 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.481834    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ed2ce2c-f93f-425e-865d-bc60cc7cc399" containerName="istio-proxy"
Jul 28 15:12:49.481882 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.481843    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ed2ce2c-f93f-425e-865d-bc60cc7cc399" containerName="istio-proxy"
Jul 28 15:12:49.484827 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.484811    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j"
Jul 28 15:12:49.487571 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.487550    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-d96a19\""
Jul 28 15:12:49.487825 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.487812    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-d96a19-dockercfg-kmb69\""
Jul 28 15:12:49.503548 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.503525    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"
Jul 28 15:12:49.507048 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.507022    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j"]
Jul 28 15:12:49.509823 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.509798    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw"]
Jul 28 15:12:49.514797 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.514774    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw"
Jul 28 15:12:49.530866 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.530831    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw"]
Jul 28 15:12:49.608980 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.608129    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istiod-ca-cert\") pod \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") "
Jul 28 15:12:49.608980 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.608177    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-token\") pod \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") "
Jul 28 15:12:49.608980 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.608229    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-data\") pod \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") "
Jul 28 15:12:49.608980 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.608268    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffxmf\" (UniqueName: \"kubernetes.io/projected/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-kube-api-access-ffxmf\") pod \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") "
Jul 28 15:12:49.608980 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.608301    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-workload-certs\") pod \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") "
Jul 28 15:12:49.608980 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.608333    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-credential-socket\") pod \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") "
Jul 28 15:12:49.608980 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.608354    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-envoy\") pod \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") "
Jul 28 15:12:49.608980 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.608415    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-podinfo\") pod \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") "
Jul 28 15:12:49.608980 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.608483    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-workload-socket\") pod \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\" (UID: \"3ed2ce2c-f93f-425e-865d-bc60cc7cc399\") "
Jul 28 15:12:49.608980 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.608642    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj8zr\" (UniqueName: \"kubernetes.io/projected/344c75f5-e576-4598-a2c9-1ecfe0a7f7c6-kube-api-access-kj8zr\") pod \"payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j\" (UID: \"344c75f5-e576-4598-a2c9-1ecfe0a7f7c6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j"
Jul 28 15:12:49.608980 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.608676    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e0bfb43-4a16-4739-9bef-b0ec4d4bb164-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw\" (UID: \"9e0bfb43-4a16-4739-9bef-b0ec4d4bb164\") " pod="openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw"
Jul 28 15:12:49.608980 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.608746    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/344c75f5-e576-4598-a2c9-1ecfe0a7f7c6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j\" (UID: \"344c75f5-e576-4598-a2c9-1ecfe0a7f7c6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j"
Jul 28 15:12:49.608980 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.608838    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5jhw\" (UniqueName: \"kubernetes.io/projected/9e0bfb43-4a16-4739-9bef-b0ec4d4bb164-kube-api-access-s5jhw\") pod \"payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw\" (UID: \"9e0bfb43-4a16-4739-9bef-b0ec4d4bb164\") " pod="openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw"
Jul 28 15:12:49.610067 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.609012    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3ed2ce2c-f93f-425e-865d-bc60cc7cc399" (UID: "3ed2ce2c-f93f-425e-865d-bc60cc7cc399"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:12:49.610067 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.609237    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3ed2ce2c-f93f-425e-865d-bc60cc7cc399" (UID: "3ed2ce2c-f93f-425e-865d-bc60cc7cc399"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:12:49.610067 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.609430    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3ed2ce2c-f93f-425e-865d-bc60cc7cc399" (UID: "3ed2ce2c-f93f-425e-865d-bc60cc7cc399"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:12:49.610067 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.609516    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3ed2ce2c-f93f-425e-865d-bc60cc7cc399" (UID: "3ed2ce2c-f93f-425e-865d-bc60cc7cc399"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:12:49.610324 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.610065    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-data" (OuterVolumeSpecName: "istio-data") pod "3ed2ce2c-f93f-425e-865d-bc60cc7cc399" (UID: "3ed2ce2c-f93f-425e-865d-bc60cc7cc399"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:12:49.614623 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.614571    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-kube-api-access-ffxmf" (OuterVolumeSpecName: "kube-api-access-ffxmf") pod "3ed2ce2c-f93f-425e-865d-bc60cc7cc399" (UID: "3ed2ce2c-f93f-425e-865d-bc60cc7cc399"). InnerVolumeSpecName "kube-api-access-ffxmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:12:49.615026 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.614846    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3ed2ce2c-f93f-425e-865d-bc60cc7cc399" (UID: "3ed2ce2c-f93f-425e-865d-bc60cc7cc399"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 15:12:49.615026 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.614874    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-token" (OuterVolumeSpecName: "istio-token") pod "3ed2ce2c-f93f-425e-865d-bc60cc7cc399" (UID: "3ed2ce2c-f93f-425e-865d-bc60cc7cc399"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:12:49.615026 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.614971    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3ed2ce2c-f93f-425e-865d-bc60cc7cc399" (UID: "3ed2ce2c-f93f-425e-865d-bc60cc7cc399"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:12:49.675598 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.675568    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"]
Jul 28 15:12:49.677138 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:12:49.677109    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33658123_a2fc_4f67_bb97_03711502555c.slice/crio-0c77b6829ef740dd928fa415ae5c1ab822a08d52666b43928856b493606138d5 WatchSource:0}: Error finding container 0c77b6829ef740dd928fa415ae5c1ab822a08d52666b43928856b493606138d5: Status 404 returned error can't find the container with id 0c77b6829ef740dd928fa415ae5c1ab822a08d52666b43928856b493606138d5
Jul 28 15:12:49.678522 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.678503    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 15:12:49.709823 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.709790    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5jhw\" (UniqueName: \"kubernetes.io/projected/9e0bfb43-4a16-4739-9bef-b0ec4d4bb164-kube-api-access-s5jhw\") pod \"payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw\" (UID: \"9e0bfb43-4a16-4739-9bef-b0ec4d4bb164\") " pod="openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw"
Jul 28 15:12:49.709978 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.709881    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj8zr\" (UniqueName: \"kubernetes.io/projected/344c75f5-e576-4598-a2c9-1ecfe0a7f7c6-kube-api-access-kj8zr\") pod \"payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j\" (UID: \"344c75f5-e576-4598-a2c9-1ecfe0a7f7c6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j"
Jul 28 15:12:49.709978 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.709909    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e0bfb43-4a16-4739-9bef-b0ec4d4bb164-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw\" (UID: \"9e0bfb43-4a16-4739-9bef-b0ec4d4bb164\") " pod="openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw"
Jul 28 15:12:49.709978 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.709962    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/344c75f5-e576-4598-a2c9-1ecfe0a7f7c6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j\" (UID: \"344c75f5-e576-4598-a2c9-1ecfe0a7f7c6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j"
Jul 28 15:12:49.710153 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.710034    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-podinfo\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:49.710153 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.710050    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-workload-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:49.710153 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.710062    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istiod-ca-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:49.710153 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.710075    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-token\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:49.710153 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.710086    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-data\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:49.710153 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.710131    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffxmf\" (UniqueName: \"kubernetes.io/projected/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-kube-api-access-ffxmf\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:49.710153 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.710144    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-workload-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:49.710401 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.710155    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-credential-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:49.710401 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.710167    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ed2ce2c-f93f-425e-865d-bc60cc7cc399-istio-envoy\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:12:49.710681 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.710659    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e0bfb43-4a16-4739-9bef-b0ec4d4bb164-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw\" (UID: \"9e0bfb43-4a16-4739-9bef-b0ec4d4bb164\") " pod="openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw"
Jul 28 15:12:49.710722 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.710693    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/344c75f5-e576-4598-a2c9-1ecfe0a7f7c6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j\" (UID: \"344c75f5-e576-4598-a2c9-1ecfe0a7f7c6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j"
Jul 28 15:12:49.720545 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.720520    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5jhw\" (UniqueName: \"kubernetes.io/projected/9e0bfb43-4a16-4739-9bef-b0ec4d4bb164-kube-api-access-s5jhw\") pod \"payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw\" (UID: \"9e0bfb43-4a16-4739-9bef-b0ec4d4bb164\") " pod="openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw"
Jul 28 15:12:49.720661 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.720564    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj8zr\" (UniqueName: \"kubernetes.io/projected/344c75f5-e576-4598-a2c9-1ecfe0a7f7c6-kube-api-access-kj8zr\") pod \"payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j\" (UID: \"344c75f5-e576-4598-a2c9-1ecfe0a7f7c6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j"
Jul 28 15:12:49.795548 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.795458    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j"
Jul 28 15:12:49.825720 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.825682    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw"
Jul 28 15:12:49.952511 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.952480    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j"]
Jul 28 15:12:49.954424 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:12:49.954394    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod344c75f5_e576_4598_a2c9_1ecfe0a7f7c6.slice/crio-a1852f6d26c777db2ae3bbceae0b7f3424b48fb5829a1ba2fdd40512eb66d934 WatchSource:0}: Error finding container a1852f6d26c777db2ae3bbceae0b7f3424b48fb5829a1ba2fdd40512eb66d934: Status 404 returned error can't find the container with id a1852f6d26c777db2ae3bbceae0b7f3424b48fb5829a1ba2fdd40512eb66d934
Jul 28 15:12:49.986980 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.986955    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw"]
Jul 28 15:12:49.993377 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:12:49.993335    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e0bfb43_4a16_4739_9bef_b0ec4d4bb164.slice/crio-55bb5e1c5782bb7dd7322507b2210e5dff5604760d48d11bbc0b8e6f696d32e6 WatchSource:0}: Error finding container 55bb5e1c5782bb7dd7322507b2210e5dff5604760d48d11bbc0b8e6f696d32e6: Status 404 returned error can't find the container with id 55bb5e1c5782bb7dd7322507b2210e5dff5604760d48d11bbc0b8e6f696d32e6
Jul 28 15:12:49.993558 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.993531    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j" event={"ID":"344c75f5-e576-4598-a2c9-1ecfe0a7f7c6","Type":"ContainerStarted","Data":"a1852f6d26c777db2ae3bbceae0b7f3424b48fb5829a1ba2fdd40512eb66d934"}
Jul 28 15:12:49.994915 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.994885    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc" event={"ID":"33658123-a2fc-4f67-bb97-03711502555c","Type":"ContainerStarted","Data":"0c77b6829ef740dd928fa415ae5c1ab822a08d52666b43928856b493606138d5"}
Jul 28 15:12:49.996652 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.996629    2584 generic.go:356] "Generic (PLEG): container finished" podID="3ed2ce2c-f93f-425e-865d-bc60cc7cc399" containerID="aada01bde3d564aeb5e6a5a517ff4a3da82850ec4727bc36ebaa3d2156abc50b" exitCode=0
Jul 28 15:12:49.996719 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.996666    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2" event={"ID":"3ed2ce2c-f93f-425e-865d-bc60cc7cc399","Type":"ContainerDied","Data":"aada01bde3d564aeb5e6a5a517ff4a3da82850ec4727bc36ebaa3d2156abc50b"}
Jul 28 15:12:49.996719 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.996691    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"
Jul 28 15:12:49.996719 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.996708    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2" event={"ID":"3ed2ce2c-f93f-425e-865d-bc60cc7cc399","Type":"ContainerDied","Data":"d81424c28de894f1483130415d38680ff3df323b26c803ce5e4198f5a103374d"}
Jul 28 15:12:49.996820 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:49.996732    2584 scope.go:117] "RemoveContainer" containerID="aada01bde3d564aeb5e6a5a517ff4a3da82850ec4727bc36ebaa3d2156abc50b"
Jul 28 15:12:50.016314 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:50.016291    2584 scope.go:117] "RemoveContainer" containerID="9fb22bcf2e033ea4d78efe280b8c63108c3f7fe6f75113fcb172a3ab5cf07266"
Jul 28 15:12:50.029277 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:50.029255    2584 scope.go:117] "RemoveContainer" containerID="aada01bde3d564aeb5e6a5a517ff4a3da82850ec4727bc36ebaa3d2156abc50b"
Jul 28 15:12:50.029604 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:12:50.029572    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aada01bde3d564aeb5e6a5a517ff4a3da82850ec4727bc36ebaa3d2156abc50b\": container with ID starting with aada01bde3d564aeb5e6a5a517ff4a3da82850ec4727bc36ebaa3d2156abc50b not found: ID does not exist" containerID="aada01bde3d564aeb5e6a5a517ff4a3da82850ec4727bc36ebaa3d2156abc50b"
Jul 28 15:12:50.029712 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:50.029616    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aada01bde3d564aeb5e6a5a517ff4a3da82850ec4727bc36ebaa3d2156abc50b"} err="failed to get container status \"aada01bde3d564aeb5e6a5a517ff4a3da82850ec4727bc36ebaa3d2156abc50b\": rpc error: code = NotFound desc = could not find container \"aada01bde3d564aeb5e6a5a517ff4a3da82850ec4727bc36ebaa3d2156abc50b\": container with ID starting with aada01bde3d564aeb5e6a5a517ff4a3da82850ec4727bc36ebaa3d2156abc50b not found: ID does not exist"
Jul 28 15:12:50.029712 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:50.029639    2584 scope.go:117] "RemoveContainer" containerID="9fb22bcf2e033ea4d78efe280b8c63108c3f7fe6f75113fcb172a3ab5cf07266"
Jul 28 15:12:50.029993 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:12:50.029955    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fb22bcf2e033ea4d78efe280b8c63108c3f7fe6f75113fcb172a3ab5cf07266\": container with ID starting with 9fb22bcf2e033ea4d78efe280b8c63108c3f7fe6f75113fcb172a3ab5cf07266 not found: ID does not exist" containerID="9fb22bcf2e033ea4d78efe280b8c63108c3f7fe6f75113fcb172a3ab5cf07266"
Jul 28 15:12:50.030243 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:50.030216    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fb22bcf2e033ea4d78efe280b8c63108c3f7fe6f75113fcb172a3ab5cf07266"} err="failed to get container status \"9fb22bcf2e033ea4d78efe280b8c63108c3f7fe6f75113fcb172a3ab5cf07266\": rpc error: code = NotFound desc = could not find container \"9fb22bcf2e033ea4d78efe280b8c63108c3f7fe6f75113fcb172a3ab5cf07266\": container with ID starting with 9fb22bcf2e033ea4d78efe280b8c63108c3f7fe6f75113fcb172a3ab5cf07266 not found: ID does not exist"
Jul 28 15:12:50.030387 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:50.030362    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"]
Jul 28 15:12:50.040905 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:50.040877    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-becc22-openshift-default-6f8fdb5ff6-56gt2"]
Jul 28 15:12:50.803579 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:50.803539    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ed2ce2c-f93f-425e-865d-bc60cc7cc399" path="/var/lib/kubelet/pods/3ed2ce2c-f93f-425e-865d-bc60cc7cc399/volumes"
Jul 28 15:12:51.004416 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:51.004374    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw" event={"ID":"9e0bfb43-4a16-4739-9bef-b0ec4d4bb164","Type":"ContainerStarted","Data":"a5f06b2f68dd7a3c0dae5334acd2b68ec5c2bc7dd3d5b06df8c14b81dc9511ba"}
Jul 28 15:12:51.004599 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:51.004440    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw" event={"ID":"9e0bfb43-4a16-4739-9bef-b0ec4d4bb164","Type":"ContainerStarted","Data":"55bb5e1c5782bb7dd7322507b2210e5dff5604760d48d11bbc0b8e6f696d32e6"}
Jul 28 15:12:51.004599 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:51.004469    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw"
Jul 28 15:12:51.006007 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:51.005976    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j" event={"ID":"344c75f5-e576-4598-a2c9-1ecfe0a7f7c6","Type":"ContainerStarted","Data":"16ef231ded6de35697512e6361926421f9e39832b368ff512bf51923365d1cdb"}
Jul 28 15:12:51.006168 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:51.006031    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j"
Jul 28 15:12:51.007463 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:51.007441    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc" event={"ID":"33658123-a2fc-4f67-bb97-03711502555c","Type":"ContainerStarted","Data":"a02e05e0261983f65e382b4a282ad7a1bd7d4530b330da9189ad7242045a8884"}
Jul 28 15:12:51.007577 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:51.007560    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"
Jul 28 15:12:51.024863 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:51.024817    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw" podStartSLOduration=2.024802481 podStartE2EDuration="2.024802481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:12:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:12:51.022184152 +0000 UTC m=+3240.823726366" watchObservedRunningTime="2026-07-28 15:12:51.024802481 +0000 UTC m=+3240.826344747"
Jul 28 15:12:51.045146 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:51.045080    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc" podStartSLOduration=2.699761075 podStartE2EDuration="3.045059915s" totalImagesPullingTime="345.29884ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:12:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:12:51.041917759 +0000 UTC m=+3240.843459975" watchObservedRunningTime="2026-07-28 15:12:51.045059915 +0000 UTC m=+3240.846602137"
Jul 28 15:12:51.064772 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:51.064688    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j" podStartSLOduration=2.064669625 podStartE2EDuration="2.064669625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:12:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:12:51.061720436 +0000 UTC m=+3240.863262651" watchObservedRunningTime="2026-07-28 15:12:51.064669625 +0000 UTC m=+3240.866211843"
Jul 28 15:12:57.018453 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:12:57.018426    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"
Jul 28 15:13:02.015931 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:02.015901    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw"
Jul 28 15:13:02.016424 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:02.015979    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j"
Jul 28 15:13:14.170561 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.170526    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"]
Jul 28 15:13:14.174258 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.174235    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.178219 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.178197    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-3edd55-openshift-default-dockercfg-vt9sf\""
Jul 28 15:13:14.191863 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.191834    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"]
Jul 28 15:13:14.237280 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.237243    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-workload-socket\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.237280 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.237278    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfqbf\" (UniqueName: \"kubernetes.io/projected/c80aaa6e-877c-4437-9b94-72105d055102-kube-api-access-pfqbf\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.237528 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.237313    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-istio-data\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.237528 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.237330    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c80aaa6e-877c-4437-9b94-72105d055102-istio-podinfo\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.237528 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.237378    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-workload-certs\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.237528 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.237418    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c80aaa6e-877c-4437-9b94-72105d055102-istiod-ca-cert\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.237528 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.237499    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-istio-envoy\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.237754 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.237577    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c80aaa6e-877c-4437-9b94-72105d055102-istio-token\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.237754 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.237604    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-credential-socket\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.338694 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.338657    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-workload-socket\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.338694 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.338699    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfqbf\" (UniqueName: \"kubernetes.io/projected/c80aaa6e-877c-4437-9b94-72105d055102-kube-api-access-pfqbf\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.338959 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.338741    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-istio-data\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.338959 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.338768    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c80aaa6e-877c-4437-9b94-72105d055102-istio-podinfo\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.338959 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.338787    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-workload-certs\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.338959 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.338816    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c80aaa6e-877c-4437-9b94-72105d055102-istiod-ca-cert\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.338959 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.338851    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-istio-envoy\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.338959 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.338908    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c80aaa6e-877c-4437-9b94-72105d055102-istio-token\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.338959 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.338935    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-credential-socket\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.339352 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.339247    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-workload-certs\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.339352 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.339323    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-istio-data\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.339438 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.339344    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-credential-socket\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.339480 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.339452    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-workload-socket\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.339580 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.339557    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c80aaa6e-877c-4437-9b94-72105d055102-istiod-ca-cert\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.341227 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.341203    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-istio-envoy\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.341321 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.341305    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c80aaa6e-877c-4437-9b94-72105d055102-istio-podinfo\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.349007 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.348983    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c80aaa6e-877c-4437-9b94-72105d055102-istio-token\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.349165 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.349078    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfqbf\" (UniqueName: \"kubernetes.io/projected/c80aaa6e-877c-4437-9b94-72105d055102-kube-api-access-pfqbf\") pod \"e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") " pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.484448 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.484357    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:14.628153 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.628126    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"]
Jul 28 15:13:14.629682 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:13:14.629656    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc80aaa6e_877c_4437_9b94_72105d055102.slice/crio-f1a99385797b3bdc1a6ed6ee385ca127f3fb66e016758035088f8c208a85102d WatchSource:0}: Error finding container f1a99385797b3bdc1a6ed6ee385ca127f3fb66e016758035088f8c208a85102d: Status 404 returned error can't find the container with id f1a99385797b3bdc1a6ed6ee385ca127f3fb66e016758035088f8c208a85102d
Jul 28 15:13:14.631822 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.631789    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 15:13:14.631888 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.631862    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 15:13:14.631939 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:14.631888    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 15:13:15.131468 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:15.131421    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq" event={"ID":"c80aaa6e-877c-4437-9b94-72105d055102","Type":"ContainerStarted","Data":"cbecb1f0bd2e4336749f3152cd2c4ef9007af690f173b9226e41d3f260600840"}
Jul 28 15:13:15.131657 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:15.131489    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq" event={"ID":"c80aaa6e-877c-4437-9b94-72105d055102","Type":"ContainerStarted","Data":"f1a99385797b3bdc1a6ed6ee385ca127f3fb66e016758035088f8c208a85102d"}
Jul 28 15:13:15.157952 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:15.157893    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq" podStartSLOduration=1.157877862 podStartE2EDuration="1.157877862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:13:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:13:15.156366316 +0000 UTC m=+3264.957908532" watchObservedRunningTime="2026-07-28 15:13:15.157877862 +0000 UTC m=+3264.959420078"
Jul 28 15:13:15.484790 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:15.484700    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:15.489665 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:15.489639    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:16.136700 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:16.136668    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:16.137725 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:16.137707    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:17.324538 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:17.324506    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"]
Jul 28 15:13:17.328304 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:17.328280    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"
Jul 28 15:13:17.332071 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:17.332046    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-3edd55-serving-cert\""
Jul 28 15:13:17.343996 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:17.343973    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"]
Jul 28 15:13:17.368755 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:17.368722    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m4rf\" (UniqueName: \"kubernetes.io/projected/a8e46d34-99a8-4ef4-a26d-36783061146b-kube-api-access-6m4rf\") pod \"maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h\" (UID: \"a8e46d34-99a8-4ef4-a26d-36783061146b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"
Jul 28 15:13:17.368892 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:17.368778    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8e46d34-99a8-4ef4-a26d-36783061146b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h\" (UID: \"a8e46d34-99a8-4ef4-a26d-36783061146b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"
Jul 28 15:13:17.368892 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:17.368849    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8e46d34-99a8-4ef4-a26d-36783061146b-maas-api-tls\") pod \"maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h\" (UID: \"a8e46d34-99a8-4ef4-a26d-36783061146b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"
Jul 28 15:13:17.470161 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:17.470123    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8e46d34-99a8-4ef4-a26d-36783061146b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h\" (UID: \"a8e46d34-99a8-4ef4-a26d-36783061146b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"
Jul 28 15:13:17.470365 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:17.470291    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8e46d34-99a8-4ef4-a26d-36783061146b-maas-api-tls\") pod \"maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h\" (UID: \"a8e46d34-99a8-4ef4-a26d-36783061146b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"
Jul 28 15:13:17.470439 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:17.470386    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m4rf\" (UniqueName: \"kubernetes.io/projected/a8e46d34-99a8-4ef4-a26d-36783061146b-kube-api-access-6m4rf\") pod \"maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h\" (UID: \"a8e46d34-99a8-4ef4-a26d-36783061146b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"
Jul 28 15:13:17.470502 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:13:17.470460    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-serving-cert: secret "maas-api-e2e-shared-b-3edd55-serving-cert" not found
Jul 28 15:13:17.470561 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:13:17.470543    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8e46d34-99a8-4ef4-a26d-36783061146b-maas-api-tls podName:a8e46d34-99a8-4ef4-a26d-36783061146b nodeName:}" failed. No retries permitted until 2026-07-28 15:13:17.970521144 +0000 UTC m=+3267.772063341 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a8e46d34-99a8-4ef4-a26d-36783061146b-maas-api-tls") pod "maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h" (UID: "a8e46d34-99a8-4ef4-a26d-36783061146b") : secret "maas-api-e2e-shared-b-3edd55-serving-cert" not found
Jul 28 15:13:17.470817 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:17.470797    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8e46d34-99a8-4ef4-a26d-36783061146b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h\" (UID: \"a8e46d34-99a8-4ef4-a26d-36783061146b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"
Jul 28 15:13:17.481929 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:17.481899    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m4rf\" (UniqueName: \"kubernetes.io/projected/a8e46d34-99a8-4ef4-a26d-36783061146b-kube-api-access-6m4rf\") pod \"maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h\" (UID: \"a8e46d34-99a8-4ef4-a26d-36783061146b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"
Jul 28 15:13:17.975215 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:17.975175    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8e46d34-99a8-4ef4-a26d-36783061146b-maas-api-tls\") pod \"maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h\" (UID: \"a8e46d34-99a8-4ef4-a26d-36783061146b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"
Jul 28 15:13:17.977775 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:17.977747    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8e46d34-99a8-4ef4-a26d-36783061146b-maas-api-tls\") pod \"maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h\" (UID: \"a8e46d34-99a8-4ef4-a26d-36783061146b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"
Jul 28 15:13:18.139382 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.139345    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d"]
Jul 28 15:13:18.142946 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.142911    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d"
Jul 28 15:13:18.146051 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.146023    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-3edd55\""
Jul 28 15:13:18.153471 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.153444    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d"]
Jul 28 15:13:18.161567 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.161538    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542"]
Jul 28 15:13:18.165576 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.165553    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542"
Jul 28 15:13:18.168221 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.168191    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-3edd55-dockercfg-7dcpd\""
Jul 28 15:13:18.177450 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.177424    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb7245e4-ecbf-465f-8203-54fb9db71ea0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d\" (UID: \"eb7245e4-ecbf-465f-8203-54fb9db71ea0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d"
Jul 28 15:13:18.177569 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.177528    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v79j\" (UniqueName: \"kubernetes.io/projected/eb7245e4-ecbf-465f-8203-54fb9db71ea0-kube-api-access-6v79j\") pod \"payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d\" (UID: \"eb7245e4-ecbf-465f-8203-54fb9db71ea0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d"
Jul 28 15:13:18.185228 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.185202    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542"]
Jul 28 15:13:18.238637 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.238550    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"
Jul 28 15:13:18.279153 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.279081    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb7245e4-ecbf-465f-8203-54fb9db71ea0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d\" (UID: \"eb7245e4-ecbf-465f-8203-54fb9db71ea0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d"
Jul 28 15:13:18.279327 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.279185    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1ab044b-c06b-46cb-b13f-b10885d9b835-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542\" (UID: \"c1ab044b-c06b-46cb-b13f-b10885d9b835\") " pod="openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542"
Jul 28 15:13:18.279327 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.279226    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v79j\" (UniqueName: \"kubernetes.io/projected/eb7245e4-ecbf-465f-8203-54fb9db71ea0-kube-api-access-6v79j\") pod \"payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d\" (UID: \"eb7245e4-ecbf-465f-8203-54fb9db71ea0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d"
Jul 28 15:13:18.279327 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.279255    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhl5p\" (UniqueName: \"kubernetes.io/projected/c1ab044b-c06b-46cb-b13f-b10885d9b835-kube-api-access-jhl5p\") pod \"payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542\" (UID: \"c1ab044b-c06b-46cb-b13f-b10885d9b835\") " pod="openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542"
Jul 28 15:13:18.280503 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.280381    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb7245e4-ecbf-465f-8203-54fb9db71ea0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d\" (UID: \"eb7245e4-ecbf-465f-8203-54fb9db71ea0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d"
Jul 28 15:13:18.290608 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.290543    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v79j\" (UniqueName: \"kubernetes.io/projected/eb7245e4-ecbf-465f-8203-54fb9db71ea0-kube-api-access-6v79j\") pod \"payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d\" (UID: \"eb7245e4-ecbf-465f-8203-54fb9db71ea0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d"
Jul 28 15:13:18.380001 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.379965    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1ab044b-c06b-46cb-b13f-b10885d9b835-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542\" (UID: \"c1ab044b-c06b-46cb-b13f-b10885d9b835\") " pod="openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542"
Jul 28 15:13:18.380436 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.380012    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhl5p\" (UniqueName: \"kubernetes.io/projected/c1ab044b-c06b-46cb-b13f-b10885d9b835-kube-api-access-jhl5p\") pod \"payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542\" (UID: \"c1ab044b-c06b-46cb-b13f-b10885d9b835\") " pod="openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542"
Jul 28 15:13:18.380808 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.380786    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1ab044b-c06b-46cb-b13f-b10885d9b835-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542\" (UID: \"c1ab044b-c06b-46cb-b13f-b10885d9b835\") " pod="openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542"
Jul 28 15:13:18.390908 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.390885    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhl5p\" (UniqueName: \"kubernetes.io/projected/c1ab044b-c06b-46cb-b13f-b10885d9b835-kube-api-access-jhl5p\") pod \"payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542\" (UID: \"c1ab044b-c06b-46cb-b13f-b10885d9b835\") " pod="openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542"
Jul 28 15:13:18.400587 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.400566    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"]
Jul 28 15:13:18.402249 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:13:18.402225    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8e46d34_99a8_4ef4_a26d_36783061146b.slice/crio-24f1fa6f24a3427eeb09cf0b80d2cae3cabcbacb66542ea730b54ee712d0cb26 WatchSource:0}: Error finding container 24f1fa6f24a3427eeb09cf0b80d2cae3cabcbacb66542ea730b54ee712d0cb26: Status 404 returned error can't find the container with id 24f1fa6f24a3427eeb09cf0b80d2cae3cabcbacb66542ea730b54ee712d0cb26
Jul 28 15:13:18.454898 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.454862    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d"
Jul 28 15:13:18.475975 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.475947    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542"
Jul 28 15:13:18.614443 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.614292    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d"]
Jul 28 15:13:18.617439 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:13:18.617399    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb7245e4_ecbf_465f_8203_54fb9db71ea0.slice/crio-22159b32b498fb90f543855516704404ce0af086c0608375afac97275fe061d5 WatchSource:0}: Error finding container 22159b32b498fb90f543855516704404ce0af086c0608375afac97275fe061d5: Status 404 returned error can't find the container with id 22159b32b498fb90f543855516704404ce0af086c0608375afac97275fe061d5
Jul 28 15:13:18.641170 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:18.641140    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542"]
Jul 28 15:13:18.642575 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:13:18.642547    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1ab044b_c06b_46cb_b13f_b10885d9b835.slice/crio-ebc747564ebf67222caa4aaf275dec0ca9cfa5cdc15f07fc66e38ef944e701b5 WatchSource:0}: Error finding container ebc747564ebf67222caa4aaf275dec0ca9cfa5cdc15f07fc66e38ef944e701b5: Status 404 returned error can't find the container with id ebc747564ebf67222caa4aaf275dec0ca9cfa5cdc15f07fc66e38ef944e701b5
Jul 28 15:13:19.153767 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:19.153711    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542" event={"ID":"c1ab044b-c06b-46cb-b13f-b10885d9b835","Type":"ContainerStarted","Data":"b81e964ba520c5fdc58dbfc1d816427e8da5a9c031a20960a0db81542f899b13"}
Jul 28 15:13:19.153957 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:19.153785    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542" event={"ID":"c1ab044b-c06b-46cb-b13f-b10885d9b835","Type":"ContainerStarted","Data":"ebc747564ebf67222caa4aaf275dec0ca9cfa5cdc15f07fc66e38ef944e701b5"}
Jul 28 15:13:19.153957 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:19.153817    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542"
Jul 28 15:13:19.155374 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:19.155347    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h" event={"ID":"a8e46d34-99a8-4ef4-a26d-36783061146b","Type":"ContainerStarted","Data":"10bd0d2e6648a00ab7ecaec1e16a4e269c100f1ec34702c6ecafd2c7a7af1eb9"}
Jul 28 15:13:19.155495 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:19.155398    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h" event={"ID":"a8e46d34-99a8-4ef4-a26d-36783061146b","Type":"ContainerStarted","Data":"24f1fa6f24a3427eeb09cf0b80d2cae3cabcbacb66542ea730b54ee712d0cb26"}
Jul 28 15:13:19.155495 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:19.155428    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"
Jul 28 15:13:19.157003 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:19.156974    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d" event={"ID":"eb7245e4-ecbf-465f-8203-54fb9db71ea0","Type":"ContainerStarted","Data":"cea1c4a34a1a6690f97919a683d6d74004b4b84d59c0a8752b3d699e696a64a4"}
Jul 28 15:13:19.157135 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:19.157030    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d"
Jul 28 15:13:19.157135 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:19.157049    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d" event={"ID":"eb7245e4-ecbf-465f-8203-54fb9db71ea0","Type":"ContainerStarted","Data":"22159b32b498fb90f543855516704404ce0af086c0608375afac97275fe061d5"}
Jul 28 15:13:19.172833 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:19.172786    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542" podStartSLOduration=1.172772056 podStartE2EDuration="1.172772056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:13:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:13:19.171058899 +0000 UTC m=+3268.972601150" watchObservedRunningTime="2026-07-28 15:13:19.172772056 +0000 UTC m=+3268.974314262"
Jul 28 15:13:19.192316 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:19.192269    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h" podStartSLOduration=1.631241063 podStartE2EDuration="2.192255194s" totalImagesPullingTime="561.014131ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:13:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:13:19.189870884 +0000 UTC m=+3268.991413100" watchObservedRunningTime="2026-07-28 15:13:19.192255194 +0000 UTC m=+3268.993797409"
Jul 28 15:13:19.208263 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:19.208217    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d" podStartSLOduration=1.208201341 podStartE2EDuration="1.208201341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:13:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:13:19.206773314 +0000 UTC m=+3269.008315528" watchObservedRunningTime="2026-07-28 15:13:19.208201341 +0000 UTC m=+3269.009743557"
Jul 28 15:13:25.167695 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:25.167665    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"
Jul 28 15:13:30.165499 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:30.165468    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d"
Jul 28 15:13:30.165982 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:30.165543    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542"
Jul 28 15:13:32.178335 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:32.178286    2584 patch_prober.go:28] interesting pod/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 15:13:32.178779 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:32.178368    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq" podUID="c80aaa6e-877c-4437-9b94-72105d055102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 15:13:33.230833 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:33.230797    2584 generic.go:356] "Generic (PLEG): container finished" podID="c80aaa6e-877c-4437-9b94-72105d055102" containerID="cbecb1f0bd2e4336749f3152cd2c4ef9007af690f173b9226e41d3f260600840" exitCode=0
Jul 28 15:13:33.231274 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:33.230873    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq" event={"ID":"c80aaa6e-877c-4437-9b94-72105d055102","Type":"ContainerDied","Data":"cbecb1f0bd2e4336749f3152cd2c4ef9007af690f173b9226e41d3f260600840"}
Jul 28 15:13:33.231356 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:33.231337    2584 scope.go:117] "RemoveContainer" containerID="cbecb1f0bd2e4336749f3152cd2c4ef9007af690f173b9226e41d3f260600840"
Jul 28 15:13:33.232407 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:33.232379    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 15:13:33.232467 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:33.232453    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 15:13:33.232555 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:33.232541    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 15:13:34.261825 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:34.261787    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq" event={"ID":"c80aaa6e-877c-4437-9b94-72105d055102","Type":"ContainerStarted","Data":"a3ab0b0cb35a60943b4243ba3f95c14ac5fa6350f5e066a8d1f2eac913ff9237"}
Jul 28 15:13:34.484596 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:34.484547    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:34.484820 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:34.484797    2584 patch_prober.go:28] interesting pod/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Jul 28 15:13:34.484868 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:34.484844    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq" podUID="c80aaa6e-877c-4437-9b94-72105d055102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Jul 28 15:13:35.485731 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:35.485690    2584 patch_prober.go:28] interesting pod/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Jul 28 15:13:35.486157 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:35.485749    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq" podUID="c80aaa6e-877c-4437-9b94-72105d055102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Jul 28 15:13:36.485587 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:36.485555    2584 patch_prober.go:28] interesting pod/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Jul 28 15:13:36.485765 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:36.485652    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq" podUID="c80aaa6e-877c-4437-9b94-72105d055102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Jul 28 15:13:37.485371 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:37.485323    2584 patch_prober.go:28] interesting pod/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Jul 28 15:13:37.485571 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:37.485423    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq" podUID="c80aaa6e-877c-4437-9b94-72105d055102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Jul 28 15:13:38.488404 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:38.488371    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:38.488896 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:38.488800    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:38.489423 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:38.489400    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:13:51.122676 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:13:51.122638    2584 scope.go:117] "RemoveContainer" containerID="534b86911e3f2678506a55c6dabf5597a802501edc0ebd9acefbe1af0446a62a"
Jul 28 15:17:51.165889 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:17:51.165853    2584 scope.go:117] "RemoveContainer" containerID="315c092d1f8c854a02eaa186727a5f7b4b1b45c4bb26788fd5df593554517d73"
Jul 28 15:23:24.096565 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.096342    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"]
Jul 28 15:23:24.100427 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.100402    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"
Jul 28 15:23:24.103755 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.103726    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-aa49c3-serving-cert\""
Jul 28 15:23:24.112883 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.112856    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"]
Jul 28 15:23:24.222939 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.222897    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/733d7759-e53e-4aec-8878-333d5eec42a8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f\" (UID: \"733d7759-e53e-4aec-8878-333d5eec42a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"
Jul 28 15:23:24.222939 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.222944    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/733d7759-e53e-4aec-8878-333d5eec42a8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f\" (UID: \"733d7759-e53e-4aec-8878-333d5eec42a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"
Jul 28 15:23:24.223216 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.223017    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb7wx\" (UniqueName: \"kubernetes.io/projected/733d7759-e53e-4aec-8878-333d5eec42a8-kube-api-access-jb7wx\") pod \"maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f\" (UID: \"733d7759-e53e-4aec-8878-333d5eec42a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"
Jul 28 15:23:24.323900 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.323856    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/733d7759-e53e-4aec-8878-333d5eec42a8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f\" (UID: \"733d7759-e53e-4aec-8878-333d5eec42a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"
Jul 28 15:23:24.324132 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.323908    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/733d7759-e53e-4aec-8878-333d5eec42a8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f\" (UID: \"733d7759-e53e-4aec-8878-333d5eec42a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"
Jul 28 15:23:24.324132 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.323975    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb7wx\" (UniqueName: \"kubernetes.io/projected/733d7759-e53e-4aec-8878-333d5eec42a8-kube-api-access-jb7wx\") pod \"maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f\" (UID: \"733d7759-e53e-4aec-8878-333d5eec42a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"
Jul 28 15:23:24.324568 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.324543    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/733d7759-e53e-4aec-8878-333d5eec42a8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f\" (UID: \"733d7759-e53e-4aec-8878-333d5eec42a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"
Jul 28 15:23:24.326490 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.326468    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/733d7759-e53e-4aec-8878-333d5eec42a8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f\" (UID: \"733d7759-e53e-4aec-8878-333d5eec42a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"
Jul 28 15:23:24.333321 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.333293    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb7wx\" (UniqueName: \"kubernetes.io/projected/733d7759-e53e-4aec-8878-333d5eec42a8-kube-api-access-jb7wx\") pod \"maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f\" (UID: \"733d7759-e53e-4aec-8878-333d5eec42a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"
Jul 28 15:23:24.412169 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.412124    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"
Jul 28 15:23:24.564385 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.564357    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"]
Jul 28 15:23:24.566445 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:23:24.566413    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod733d7759_e53e_4aec_8878_333d5eec42a8.slice/crio-9d55825126e488ce17332629183b71cb080ab2eeaea137fcc0b181281f6eaebd WatchSource:0}: Error finding container 9d55825126e488ce17332629183b71cb080ab2eeaea137fcc0b181281f6eaebd: Status 404 returned error can't find the container with id 9d55825126e488ce17332629183b71cb080ab2eeaea137fcc0b181281f6eaebd
Jul 28 15:23:24.567789 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.567770    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 15:23:24.818494 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.818415    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl"]
Jul 28 15:23:24.823309 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.823289    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl"
Jul 28 15:23:24.835199 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.835174    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-aa49c3\""
Jul 28 15:23:24.835522 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.835507    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-aa49c3-dockercfg-4fcgv\""
Jul 28 15:23:24.850076 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.850041    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl"]
Jul 28 15:23:24.930321 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.930272    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98a66db1-31b6-47aa-b879-1049d20e9877-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl\" (UID: \"98a66db1-31b6-47aa-b879-1049d20e9877\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl"
Jul 28 15:23:24.930542 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:24.930356    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh8f7\" (UniqueName: \"kubernetes.io/projected/98a66db1-31b6-47aa-b879-1049d20e9877-kube-api-access-rh8f7\") pod \"payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl\" (UID: \"98a66db1-31b6-47aa-b879-1049d20e9877\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl"
Jul 28 15:23:25.031684 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:25.031645    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh8f7\" (UniqueName: \"kubernetes.io/projected/98a66db1-31b6-47aa-b879-1049d20e9877-kube-api-access-rh8f7\") pod \"payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl\" (UID: \"98a66db1-31b6-47aa-b879-1049d20e9877\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl"
Jul 28 15:23:25.031869 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:25.031815    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98a66db1-31b6-47aa-b879-1049d20e9877-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl\" (UID: \"98a66db1-31b6-47aa-b879-1049d20e9877\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl"
Jul 28 15:23:25.032567 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:25.032544    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98a66db1-31b6-47aa-b879-1049d20e9877-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl\" (UID: \"98a66db1-31b6-47aa-b879-1049d20e9877\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl"
Jul 28 15:23:25.043232 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:25.043203    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh8f7\" (UniqueName: \"kubernetes.io/projected/98a66db1-31b6-47aa-b879-1049d20e9877-kube-api-access-rh8f7\") pod \"payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl\" (UID: \"98a66db1-31b6-47aa-b879-1049d20e9877\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl"
Jul 28 15:23:25.134422 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:25.134377    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl"
Jul 28 15:23:25.240213 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:25.240160    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f" event={"ID":"733d7759-e53e-4aec-8878-333d5eec42a8","Type":"ContainerStarted","Data":"5a8953da4b43f2561fa1f24f9235df2f56388208c4d1d2f663207ac51006d1e2"}
Jul 28 15:23:25.240532 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:25.240509    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"
Jul 28 15:23:25.240697 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:25.240666    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f" event={"ID":"733d7759-e53e-4aec-8878-333d5eec42a8","Type":"ContainerStarted","Data":"9d55825126e488ce17332629183b71cb080ab2eeaea137fcc0b181281f6eaebd"}
Jul 28 15:23:25.262431 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:25.262359    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f" podStartSLOduration=0.71326717 podStartE2EDuration="1.262334473s" totalImagesPullingTime="549.067303ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:23:25.261611069 +0000 UTC m=+3875.063153284" watchObservedRunningTime="2026-07-28 15:23:25.262334473 +0000 UTC m=+3875.063876670"
Jul 28 15:23:25.297451 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:25.297424    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl"]
Jul 28 15:23:25.299704 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:23:25.299672    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98a66db1_31b6_47aa_b879_1049d20e9877.slice/crio-8df64f68a90e14804b56baddf9449e21f3d52dc643c28952c2334113ca196293 WatchSource:0}: Error finding container 8df64f68a90e14804b56baddf9449e21f3d52dc643c28952c2334113ca196293: Status 404 returned error can't find the container with id 8df64f68a90e14804b56baddf9449e21f3d52dc643c28952c2334113ca196293
Jul 28 15:23:26.252900 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:26.252855    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl" event={"ID":"98a66db1-31b6-47aa-b879-1049d20e9877","Type":"ContainerStarted","Data":"e2c25f3fc1a199adea003b4b9f5017d431614f3bd93c6481274894344348bd68"}
Jul 28 15:23:26.253417 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:26.252941    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl" event={"ID":"98a66db1-31b6-47aa-b879-1049d20e9877","Type":"ContainerStarted","Data":"8df64f68a90e14804b56baddf9449e21f3d52dc643c28952c2334113ca196293"}
Jul 28 15:23:26.253417 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:26.253053    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl"
Jul 28 15:23:26.277795 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:26.277740    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl" podStartSLOduration=2.277724461 podStartE2EDuration="2.277724461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:23:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:23:26.274478777 +0000 UTC m=+3876.076020992" watchObservedRunningTime="2026-07-28 15:23:26.277724461 +0000 UTC m=+3876.079266676"
Jul 28 15:23:32.264113 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:32.264060    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"
Jul 28 15:23:37.261167 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:37.261136    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl"
Jul 28 15:23:48.897985 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:48.897945    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"]
Jul 28 15:23:48.901275 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:48.901256    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"
Jul 28 15:23:48.905343 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:48.905322    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-034645-serving-cert\""
Jul 28 15:23:48.921308 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:48.921268    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"]
Jul 28 15:23:48.973955 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:48.973906    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/000c0b01-1123-4774-9aeb-f64eb32e595a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k\" (UID: \"000c0b01-1123-4774-9aeb-f64eb32e595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"
Jul 28 15:23:48.974188 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:48.973973    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tflvr\" (UniqueName: \"kubernetes.io/projected/000c0b01-1123-4774-9aeb-f64eb32e595a-kube-api-access-tflvr\") pod \"maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k\" (UID: \"000c0b01-1123-4774-9aeb-f64eb32e595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"
Jul 28 15:23:48.974188 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:48.974033    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/000c0b01-1123-4774-9aeb-f64eb32e595a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k\" (UID: \"000c0b01-1123-4774-9aeb-f64eb32e595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"
Jul 28 15:23:49.075208 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:49.075166    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/000c0b01-1123-4774-9aeb-f64eb32e595a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k\" (UID: \"000c0b01-1123-4774-9aeb-f64eb32e595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"
Jul 28 15:23:49.075430 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:49.075277    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/000c0b01-1123-4774-9aeb-f64eb32e595a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k\" (UID: \"000c0b01-1123-4774-9aeb-f64eb32e595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"
Jul 28 15:23:49.075430 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:49.075311    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tflvr\" (UniqueName: \"kubernetes.io/projected/000c0b01-1123-4774-9aeb-f64eb32e595a-kube-api-access-tflvr\") pod \"maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k\" (UID: \"000c0b01-1123-4774-9aeb-f64eb32e595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"
Jul 28 15:23:49.075430 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:23:49.075342    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-serving-cert: secret "maas-api-e2e-ipp-b-034645-serving-cert" not found
Jul 28 15:23:49.075606 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:23:49.075433    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/000c0b01-1123-4774-9aeb-f64eb32e595a-maas-api-tls podName:000c0b01-1123-4774-9aeb-f64eb32e595a nodeName:}" failed. No retries permitted until 2026-07-28 15:23:49.575411214 +0000 UTC m=+3899.376953407 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/000c0b01-1123-4774-9aeb-f64eb32e595a-maas-api-tls") pod "maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k" (UID: "000c0b01-1123-4774-9aeb-f64eb32e595a") : secret "maas-api-e2e-ipp-b-034645-serving-cert" not found
Jul 28 15:23:49.075927 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:49.075908    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/000c0b01-1123-4774-9aeb-f64eb32e595a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k\" (UID: \"000c0b01-1123-4774-9aeb-f64eb32e595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"
Jul 28 15:23:49.091114 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:49.091071    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tflvr\" (UniqueName: \"kubernetes.io/projected/000c0b01-1123-4774-9aeb-f64eb32e595a-kube-api-access-tflvr\") pod \"maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k\" (UID: \"000c0b01-1123-4774-9aeb-f64eb32e595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"
Jul 28 15:23:49.580591 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:49.580555    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/000c0b01-1123-4774-9aeb-f64eb32e595a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k\" (UID: \"000c0b01-1123-4774-9aeb-f64eb32e595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"
Jul 28 15:23:49.583084 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:49.583053    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/000c0b01-1123-4774-9aeb-f64eb32e595a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k\" (UID: \"000c0b01-1123-4774-9aeb-f64eb32e595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"
Jul 28 15:23:49.811308 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:49.811270    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"
Jul 28 15:23:49.920465 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:49.920427    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt"]
Jul 28 15:23:49.924344 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:49.924322    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt"
Jul 28 15:23:49.927284 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:49.927259    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-034645\""
Jul 28 15:23:49.939851 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:49.939822    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt"]
Jul 28 15:23:49.977367 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:49.977334    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"]
Jul 28 15:23:49.980797 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:23:49.980769    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod000c0b01_1123_4774_9aeb_f64eb32e595a.slice/crio-e21cd559d3f068f95632cae4180c7b812e898a6fea31b9a11735c85e8ff7ac54 WatchSource:0}: Error finding container e21cd559d3f068f95632cae4180c7b812e898a6fea31b9a11735c85e8ff7ac54: Status 404 returned error can't find the container with id e21cd559d3f068f95632cae4180c7b812e898a6fea31b9a11735c85e8ff7ac54
Jul 28 15:23:49.984229 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:49.984202    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb5hd\" (UniqueName: \"kubernetes.io/projected/279bbc3f-03c7-4f6f-a30b-daf6940e7c92-kube-api-access-vb5hd\") pod \"payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt\" (UID: \"279bbc3f-03c7-4f6f-a30b-daf6940e7c92\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt"
Jul 28 15:23:49.984334 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:49.984256    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/279bbc3f-03c7-4f6f-a30b-daf6940e7c92-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt\" (UID: \"279bbc3f-03c7-4f6f-a30b-daf6940e7c92\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt"
Jul 28 15:23:50.086074 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:50.086028    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb5hd\" (UniqueName: \"kubernetes.io/projected/279bbc3f-03c7-4f6f-a30b-daf6940e7c92-kube-api-access-vb5hd\") pod \"payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt\" (UID: \"279bbc3f-03c7-4f6f-a30b-daf6940e7c92\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt"
Jul 28 15:23:50.086318 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:50.086149    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/279bbc3f-03c7-4f6f-a30b-daf6940e7c92-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt\" (UID: \"279bbc3f-03c7-4f6f-a30b-daf6940e7c92\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt"
Jul 28 15:23:50.087033 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:50.086994    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/279bbc3f-03c7-4f6f-a30b-daf6940e7c92-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt\" (UID: \"279bbc3f-03c7-4f6f-a30b-daf6940e7c92\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt"
Jul 28 15:23:50.096167 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:50.096131    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb5hd\" (UniqueName: \"kubernetes.io/projected/279bbc3f-03c7-4f6f-a30b-daf6940e7c92-kube-api-access-vb5hd\") pod \"payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt\" (UID: \"279bbc3f-03c7-4f6f-a30b-daf6940e7c92\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt"
Jul 28 15:23:50.238581 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:50.238483    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt"
Jul 28 15:23:50.378289 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:50.378251    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k" event={"ID":"000c0b01-1123-4774-9aeb-f64eb32e595a","Type":"ContainerStarted","Data":"e21cd559d3f068f95632cae4180c7b812e898a6fea31b9a11735c85e8ff7ac54"}
Jul 28 15:23:50.397401 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:50.397358    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt"]
Jul 28 15:23:50.400369 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:23:50.400337    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod279bbc3f_03c7_4f6f_a30b_daf6940e7c92.slice/crio-ed6880e94d6f27f2983c50ded279e64d573db18bb3343803e3894501533ec402 WatchSource:0}: Error finding container ed6880e94d6f27f2983c50ded279e64d573db18bb3343803e3894501533ec402: Status 404 returned error can't find the container with id ed6880e94d6f27f2983c50ded279e64d573db18bb3343803e3894501533ec402
Jul 28 15:23:51.385042 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:51.384998    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k" event={"ID":"000c0b01-1123-4774-9aeb-f64eb32e595a","Type":"ContainerStarted","Data":"15782a683b95248926d5c20acf22dd31f009807ed1e5be0e771283345a81b3b0"}
Jul 28 15:23:51.385559 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:51.385210    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"
Jul 28 15:23:51.386556 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:51.386530    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt" event={"ID":"279bbc3f-03c7-4f6f-a30b-daf6940e7c92","Type":"ContainerStarted","Data":"2919fcda8f38279525cc9e7803562e1cc6f78dcc2fd24aadcc7f4613c491ca34"}
Jul 28 15:23:51.386686 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:51.386583    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt"
Jul 28 15:23:51.386686 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:51.386609    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt" event={"ID":"279bbc3f-03c7-4f6f-a30b-daf6940e7c92","Type":"ContainerStarted","Data":"ed6880e94d6f27f2983c50ded279e64d573db18bb3343803e3894501533ec402"}
Jul 28 15:23:51.402693 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:51.402639    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k" podStartSLOduration=2.866010443 podStartE2EDuration="3.402622633s" totalImagesPullingTime="536.61219ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:23:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:23:51.401204856 +0000 UTC m=+3901.202747073" watchObservedRunningTime="2026-07-28 15:23:51.402622633 +0000 UTC m=+3901.204164848"
Jul 28 15:23:51.419149 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:51.419080    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt" podStartSLOduration=2.41905964 podStartE2EDuration="2.41905964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:23:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:23:51.418242135 +0000 UTC m=+3901.219784351" watchObservedRunningTime="2026-07-28 15:23:51.41905964 +0000 UTC m=+3901.220601856"
Jul 28 15:23:57.396817 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:23:57.396788    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"
Jul 28 15:24:02.394526 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:24:02.394487    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt"
Jul 28 15:24:13.278412 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:24:13.278371    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-2hwhh_5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2/payload-processing/0.log"
Jul 28 15:24:58.424328 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:24:58.424242    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl_98a66db1-31b6-47aa-b879-1049d20e9877/payload-processing/0.log"
Jul 28 15:24:58.549626 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:24:58.549592    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-2hwhh_5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2/payload-processing/0.log"
Jul 28 15:25:06.536257 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:06.536215    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"]
Jul 28 15:25:06.540764 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:06.540737    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"
Jul 28 15:25:06.543917 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:06.543890    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-9498a6-serving-cert\""
Jul 28 15:25:06.561060 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:06.561027    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"]
Jul 28 15:25:06.645635 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:06.645591    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgn5d\" (UniqueName: \"kubernetes.io/projected/5c894837-74a6-4b3b-822b-67582d218ec1-kube-api-access-tgn5d\") pod \"maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg\" (UID: \"5c894837-74a6-4b3b-822b-67582d218ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"
Jul 28 15:25:06.646011 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:06.645982    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c894837-74a6-4b3b-822b-67582d218ec1-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg\" (UID: \"5c894837-74a6-4b3b-822b-67582d218ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"
Jul 28 15:25:06.646168 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:06.646143    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c894837-74a6-4b3b-822b-67582d218ec1-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg\" (UID: \"5c894837-74a6-4b3b-822b-67582d218ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"
Jul 28 15:25:06.747518 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:06.747477    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgn5d\" (UniqueName: \"kubernetes.io/projected/5c894837-74a6-4b3b-822b-67582d218ec1-kube-api-access-tgn5d\") pod \"maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg\" (UID: \"5c894837-74a6-4b3b-822b-67582d218ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"
Jul 28 15:25:06.747716 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:06.747627    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c894837-74a6-4b3b-822b-67582d218ec1-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg\" (UID: \"5c894837-74a6-4b3b-822b-67582d218ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"
Jul 28 15:25:06.747716 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:06.747650    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c894837-74a6-4b3b-822b-67582d218ec1-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg\" (UID: \"5c894837-74a6-4b3b-822b-67582d218ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"
Jul 28 15:25:06.748413 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:06.748388    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c894837-74a6-4b3b-822b-67582d218ec1-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg\" (UID: \"5c894837-74a6-4b3b-822b-67582d218ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"
Jul 28 15:25:06.750162 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:06.750142    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c894837-74a6-4b3b-822b-67582d218ec1-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg\" (UID: \"5c894837-74a6-4b3b-822b-67582d218ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"
Jul 28 15:25:06.759338 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:06.759309    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgn5d\" (UniqueName: \"kubernetes.io/projected/5c894837-74a6-4b3b-822b-67582d218ec1-kube-api-access-tgn5d\") pod \"maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg\" (UID: \"5c894837-74a6-4b3b-822b-67582d218ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"
Jul 28 15:25:06.852053 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:06.852020    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"
Jul 28 15:25:07.011956 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:07.011928    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"]
Jul 28 15:25:07.014642 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:25:07.014609    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c894837_74a6_4b3b_822b_67582d218ec1.slice/crio-cc153dd74d919aa2a40a961bd8ac6df7f596aae09744b3bcb26910bbc76bb163 WatchSource:0}: Error finding container cc153dd74d919aa2a40a961bd8ac6df7f596aae09744b3bcb26910bbc76bb163: Status 404 returned error can't find the container with id cc153dd74d919aa2a40a961bd8ac6df7f596aae09744b3bcb26910bbc76bb163
Jul 28 15:25:07.816893 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:07.816797    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg" event={"ID":"5c894837-74a6-4b3b-822b-67582d218ec1","Type":"ContainerStarted","Data":"4b93268061211b9932180ccdf655107769decb157ba9859cddf2061947077d2d"}
Jul 28 15:25:07.816893 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:07.816866    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg" event={"ID":"5c894837-74a6-4b3b-822b-67582d218ec1","Type":"ContainerStarted","Data":"cc153dd74d919aa2a40a961bd8ac6df7f596aae09744b3bcb26910bbc76bb163"}
Jul 28 15:25:07.817329 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:07.817030    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"
Jul 28 15:25:07.852555 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:07.852491    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg" podStartSLOduration=1.406206798 podStartE2EDuration="1.852465684s" totalImagesPullingTime="446.258886ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:25:07.848916715 +0000 UTC m=+3977.650458930" watchObservedRunningTime="2026-07-28 15:25:07.852465684 +0000 UTC m=+3977.654007901"
Jul 28 15:25:13.827111 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:13.827057    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"
Jul 28 15:25:45.013508 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:45.013471    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"]
Jul 28 15:25:45.014178 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:45.013727    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg" podUID="5c894837-74a6-4b3b-822b-67582d218ec1" containerName="maas-api" containerID="cri-o://4b93268061211b9932180ccdf655107769decb157ba9859cddf2061947077d2d" gracePeriod=30
Jul 28 15:25:45.298676 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:45.298642    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"
Jul 28 15:25:45.449504 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:45.449472    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c894837-74a6-4b3b-822b-67582d218ec1-openshift-service-ca\") pod \"5c894837-74a6-4b3b-822b-67582d218ec1\" (UID: \"5c894837-74a6-4b3b-822b-67582d218ec1\") "
Jul 28 15:25:45.449675 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:45.449543    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgn5d\" (UniqueName: \"kubernetes.io/projected/5c894837-74a6-4b3b-822b-67582d218ec1-kube-api-access-tgn5d\") pod \"5c894837-74a6-4b3b-822b-67582d218ec1\" (UID: \"5c894837-74a6-4b3b-822b-67582d218ec1\") "
Jul 28 15:25:45.449675 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:45.449598    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c894837-74a6-4b3b-822b-67582d218ec1-maas-api-tls\") pod \"5c894837-74a6-4b3b-822b-67582d218ec1\" (UID: \"5c894837-74a6-4b3b-822b-67582d218ec1\") "
Jul 28 15:25:45.449911 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:45.449889    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c894837-74a6-4b3b-822b-67582d218ec1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5c894837-74a6-4b3b-822b-67582d218ec1" (UID: "5c894837-74a6-4b3b-822b-67582d218ec1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:25:45.450011 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:45.449992    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c894837-74a6-4b3b-822b-67582d218ec1-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:25:45.451718 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:45.451696    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c894837-74a6-4b3b-822b-67582d218ec1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5c894837-74a6-4b3b-822b-67582d218ec1" (UID: "5c894837-74a6-4b3b-822b-67582d218ec1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:25:45.451808 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:45.451788    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c894837-74a6-4b3b-822b-67582d218ec1-kube-api-access-tgn5d" (OuterVolumeSpecName: "kube-api-access-tgn5d") pod "5c894837-74a6-4b3b-822b-67582d218ec1" (UID: "5c894837-74a6-4b3b-822b-67582d218ec1"). InnerVolumeSpecName "kube-api-access-tgn5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:25:45.551487 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:45.551385    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgn5d\" (UniqueName: \"kubernetes.io/projected/5c894837-74a6-4b3b-822b-67582d218ec1-kube-api-access-tgn5d\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:25:45.551487 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:45.551423    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c894837-74a6-4b3b-822b-67582d218ec1-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:25:46.024663 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:46.024633    2584 generic.go:356] "Generic (PLEG): container finished" podID="5c894837-74a6-4b3b-822b-67582d218ec1" containerID="4b93268061211b9932180ccdf655107769decb157ba9859cddf2061947077d2d" exitCode=0
Jul 28 15:25:46.025135 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:46.024677    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"
Jul 28 15:25:46.025135 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:46.024711    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg" event={"ID":"5c894837-74a6-4b3b-822b-67582d218ec1","Type":"ContainerDied","Data":"4b93268061211b9932180ccdf655107769decb157ba9859cddf2061947077d2d"}
Jul 28 15:25:46.025135 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:46.024774    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg" event={"ID":"5c894837-74a6-4b3b-822b-67582d218ec1","Type":"ContainerDied","Data":"cc153dd74d919aa2a40a961bd8ac6df7f596aae09744b3bcb26910bbc76bb163"}
Jul 28 15:25:46.025135 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:46.024802    2584 scope.go:117] "RemoveContainer" containerID="4b93268061211b9932180ccdf655107769decb157ba9859cddf2061947077d2d"
Jul 28 15:25:46.034586 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:46.034564    2584 scope.go:117] "RemoveContainer" containerID="4b93268061211b9932180ccdf655107769decb157ba9859cddf2061947077d2d"
Jul 28 15:25:46.034864 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:25:46.034844    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b93268061211b9932180ccdf655107769decb157ba9859cddf2061947077d2d\": container with ID starting with 4b93268061211b9932180ccdf655107769decb157ba9859cddf2061947077d2d not found: ID does not exist" containerID="4b93268061211b9932180ccdf655107769decb157ba9859cddf2061947077d2d"
Jul 28 15:25:46.034914 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:46.034873    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b93268061211b9932180ccdf655107769decb157ba9859cddf2061947077d2d"} err="failed to get container status \"4b93268061211b9932180ccdf655107769decb157ba9859cddf2061947077d2d\": rpc error: code = NotFound desc = could not find container \"4b93268061211b9932180ccdf655107769decb157ba9859cddf2061947077d2d\": container with ID starting with 4b93268061211b9932180ccdf655107769decb157ba9859cddf2061947077d2d not found: ID does not exist"
Jul 28 15:25:46.054275 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:46.054233    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"]
Jul 28 15:25:46.065502 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:46.065464    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9498a6-7c74f7994f-8cqkg"]
Jul 28 15:25:46.797228 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:25:46.797193    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c894837-74a6-4b3b-822b-67582d218ec1" path="/var/lib/kubelet/pods/5c894837-74a6-4b3b-822b-67582d218ec1/volumes"
Jul 28 15:26:13.512287 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:13.512199    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"]
Jul 28 15:26:13.512787 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:13.512470    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f" podUID="733d7759-e53e-4aec-8878-333d5eec42a8" containerName="maas-api" containerID="cri-o://5a8953da4b43f2561fa1f24f9235df2f56388208c4d1d2f663207ac51006d1e2" gracePeriod=30
Jul 28 15:26:13.664645 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:13.664612    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl"]
Jul 28 15:26:13.665238 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:13.664961    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl" podUID="98a66db1-31b6-47aa-b879-1049d20e9877" containerName="payload-processing" containerID="cri-o://e2c25f3fc1a199adea003b4b9f5017d431614f3bd93c6481274894344348bd68" gracePeriod=30
Jul 28 15:26:13.810533 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:13.810503    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"
Jul 28 15:26:13.939865 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:13.939829    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jb7wx\" (UniqueName: \"kubernetes.io/projected/733d7759-e53e-4aec-8878-333d5eec42a8-kube-api-access-jb7wx\") pod \"733d7759-e53e-4aec-8878-333d5eec42a8\" (UID: \"733d7759-e53e-4aec-8878-333d5eec42a8\") "
Jul 28 15:26:13.940035 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:13.939947    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/733d7759-e53e-4aec-8878-333d5eec42a8-openshift-service-ca\") pod \"733d7759-e53e-4aec-8878-333d5eec42a8\" (UID: \"733d7759-e53e-4aec-8878-333d5eec42a8\") "
Jul 28 15:26:13.940035 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:13.940025    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/733d7759-e53e-4aec-8878-333d5eec42a8-maas-api-tls\") pod \"733d7759-e53e-4aec-8878-333d5eec42a8\" (UID: \"733d7759-e53e-4aec-8878-333d5eec42a8\") "
Jul 28 15:26:13.940517 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:13.940458    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/733d7759-e53e-4aec-8878-333d5eec42a8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "733d7759-e53e-4aec-8878-333d5eec42a8" (UID: "733d7759-e53e-4aec-8878-333d5eec42a8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:26:13.942961 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:13.942929    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/733d7759-e53e-4aec-8878-333d5eec42a8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "733d7759-e53e-4aec-8878-333d5eec42a8" (UID: "733d7759-e53e-4aec-8878-333d5eec42a8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:26:13.943197 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:13.943176    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/733d7759-e53e-4aec-8878-333d5eec42a8-kube-api-access-jb7wx" (OuterVolumeSpecName: "kube-api-access-jb7wx") pod "733d7759-e53e-4aec-8878-333d5eec42a8" (UID: "733d7759-e53e-4aec-8878-333d5eec42a8"). InnerVolumeSpecName "kube-api-access-jb7wx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:26:13.961749 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:13.961722    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl"
Jul 28 15:26:14.041625 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.041531    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rh8f7\" (UniqueName: \"kubernetes.io/projected/98a66db1-31b6-47aa-b879-1049d20e9877-kube-api-access-rh8f7\") pod \"98a66db1-31b6-47aa-b879-1049d20e9877\" (UID: \"98a66db1-31b6-47aa-b879-1049d20e9877\") "
Jul 28 15:26:14.041625 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.041591    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98a66db1-31b6-47aa-b879-1049d20e9877-plugins-config-volume\") pod \"98a66db1-31b6-47aa-b879-1049d20e9877\" (UID: \"98a66db1-31b6-47aa-b879-1049d20e9877\") "
Jul 28 15:26:14.041833 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.041752    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/733d7759-e53e-4aec-8878-333d5eec42a8-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:26:14.041833 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.041763    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/733d7759-e53e-4aec-8878-333d5eec42a8-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:26:14.041833 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.041772    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jb7wx\" (UniqueName: \"kubernetes.io/projected/733d7759-e53e-4aec-8878-333d5eec42a8-kube-api-access-jb7wx\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:26:14.041975 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.041953    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98a66db1-31b6-47aa-b879-1049d20e9877-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "98a66db1-31b6-47aa-b879-1049d20e9877" (UID: "98a66db1-31b6-47aa-b879-1049d20e9877"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:26:14.043642 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.043620    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98a66db1-31b6-47aa-b879-1049d20e9877-kube-api-access-rh8f7" (OuterVolumeSpecName: "kube-api-access-rh8f7") pod "98a66db1-31b6-47aa-b879-1049d20e9877" (UID: "98a66db1-31b6-47aa-b879-1049d20e9877"). InnerVolumeSpecName "kube-api-access-rh8f7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:26:14.142480 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.142438    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rh8f7\" (UniqueName: \"kubernetes.io/projected/98a66db1-31b6-47aa-b879-1049d20e9877-kube-api-access-rh8f7\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:26:14.142480 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.142473    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98a66db1-31b6-47aa-b879-1049d20e9877-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:26:14.171884 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.171843    2584 generic.go:356] "Generic (PLEG): container finished" podID="733d7759-e53e-4aec-8878-333d5eec42a8" containerID="5a8953da4b43f2561fa1f24f9235df2f56388208c4d1d2f663207ac51006d1e2" exitCode=0
Jul 28 15:26:14.172066 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.171908    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"
Jul 28 15:26:14.172066 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.171931    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f" event={"ID":"733d7759-e53e-4aec-8878-333d5eec42a8","Type":"ContainerDied","Data":"5a8953da4b43f2561fa1f24f9235df2f56388208c4d1d2f663207ac51006d1e2"}
Jul 28 15:26:14.172066 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.172010    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f" event={"ID":"733d7759-e53e-4aec-8878-333d5eec42a8","Type":"ContainerDied","Data":"9d55825126e488ce17332629183b71cb080ab2eeaea137fcc0b181281f6eaebd"}
Jul 28 15:26:14.172066 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.172038    2584 scope.go:117] "RemoveContainer" containerID="5a8953da4b43f2561fa1f24f9235df2f56388208c4d1d2f663207ac51006d1e2"
Jul 28 15:26:14.173271 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.173246    2584 generic.go:356] "Generic (PLEG): container finished" podID="98a66db1-31b6-47aa-b879-1049d20e9877" containerID="e2c25f3fc1a199adea003b4b9f5017d431614f3bd93c6481274894344348bd68" exitCode=0
Jul 28 15:26:14.173339 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.173284    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl"
Jul 28 15:26:14.173339 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.173299    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl" event={"ID":"98a66db1-31b6-47aa-b879-1049d20e9877","Type":"ContainerDied","Data":"e2c25f3fc1a199adea003b4b9f5017d431614f3bd93c6481274894344348bd68"}
Jul 28 15:26:14.173416 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.173343    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl" event={"ID":"98a66db1-31b6-47aa-b879-1049d20e9877","Type":"ContainerDied","Data":"8df64f68a90e14804b56baddf9449e21f3d52dc643c28952c2334113ca196293"}
Jul 28 15:26:14.182514 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.182491    2584 scope.go:117] "RemoveContainer" containerID="5a8953da4b43f2561fa1f24f9235df2f56388208c4d1d2f663207ac51006d1e2"
Jul 28 15:26:14.182824 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:26:14.182806    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a8953da4b43f2561fa1f24f9235df2f56388208c4d1d2f663207ac51006d1e2\": container with ID starting with 5a8953da4b43f2561fa1f24f9235df2f56388208c4d1d2f663207ac51006d1e2 not found: ID does not exist" containerID="5a8953da4b43f2561fa1f24f9235df2f56388208c4d1d2f663207ac51006d1e2"
Jul 28 15:26:14.182873 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.182835    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a8953da4b43f2561fa1f24f9235df2f56388208c4d1d2f663207ac51006d1e2"} err="failed to get container status \"5a8953da4b43f2561fa1f24f9235df2f56388208c4d1d2f663207ac51006d1e2\": rpc error: code = NotFound desc = could not find container \"5a8953da4b43f2561fa1f24f9235df2f56388208c4d1d2f663207ac51006d1e2\": container with ID starting with 5a8953da4b43f2561fa1f24f9235df2f56388208c4d1d2f663207ac51006d1e2 not found: ID does not exist"
Jul 28 15:26:14.182873 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.182853    2584 scope.go:117] "RemoveContainer" containerID="e2c25f3fc1a199adea003b4b9f5017d431614f3bd93c6481274894344348bd68"
Jul 28 15:26:14.192454 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.192433    2584 scope.go:117] "RemoveContainer" containerID="e2c25f3fc1a199adea003b4b9f5017d431614f3bd93c6481274894344348bd68"
Jul 28 15:26:14.192725 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:26:14.192705    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2c25f3fc1a199adea003b4b9f5017d431614f3bd93c6481274894344348bd68\": container with ID starting with e2c25f3fc1a199adea003b4b9f5017d431614f3bd93c6481274894344348bd68 not found: ID does not exist" containerID="e2c25f3fc1a199adea003b4b9f5017d431614f3bd93c6481274894344348bd68"
Jul 28 15:26:14.192779 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.192733    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2c25f3fc1a199adea003b4b9f5017d431614f3bd93c6481274894344348bd68"} err="failed to get container status \"e2c25f3fc1a199adea003b4b9f5017d431614f3bd93c6481274894344348bd68\": rpc error: code = NotFound desc = could not find container \"e2c25f3fc1a199adea003b4b9f5017d431614f3bd93c6481274894344348bd68\": container with ID starting with e2c25f3fc1a199adea003b4b9f5017d431614f3bd93c6481274894344348bd68 not found: ID does not exist"
Jul 28 15:26:14.198563 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.198535    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl"]
Jul 28 15:26:14.202381 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.202358    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-aa49c3-68cccb658d-xbqpl"]
Jul 28 15:26:14.216353 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.216322    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"]
Jul 28 15:26:14.220368 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.220343    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-aa49c3-575bd5564d-97q6f"]
Jul 28 15:26:14.797328 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.797247    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="733d7759-e53e-4aec-8878-333d5eec42a8" path="/var/lib/kubelet/pods/733d7759-e53e-4aec-8878-333d5eec42a8/volumes"
Jul 28 15:26:14.797708 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:26:14.797629    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98a66db1-31b6-47aa-b879-1049d20e9877" path="/var/lib/kubelet/pods/98a66db1-31b6-47aa-b879-1049d20e9877/volumes"
Jul 28 15:27:12.377136 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.377104    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"]
Jul 28 15:27:12.377688 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.377355    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k" podUID="000c0b01-1123-4774-9aeb-f64eb32e595a" containerName="maas-api" containerID="cri-o://15782a683b95248926d5c20acf22dd31f009807ed1e5be0e771283345a81b3b0" gracePeriod=30
Jul 28 15:27:12.392837 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.392799    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k" podUID="000c0b01-1123-4774-9aeb-f64eb32e595a" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.88:8443/health\": dial tcp 10.134.0.88:8443: connect: connection refused"
Jul 28 15:27:12.471969 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.471940    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt"]
Jul 28 15:27:12.472251 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.472175    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt" podUID="279bbc3f-03c7-4f6f-a30b-daf6940e7c92" containerName="payload-pre-processing" containerID="cri-o://2919fcda8f38279525cc9e7803562e1cc6f78dcc2fd24aadcc7f4613c491ca34" gracePeriod=30
Jul 28 15:27:12.672589 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.672502    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"
Jul 28 15:27:12.733203 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.733172    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/000c0b01-1123-4774-9aeb-f64eb32e595a-maas-api-tls\") pod \"000c0b01-1123-4774-9aeb-f64eb32e595a\" (UID: \"000c0b01-1123-4774-9aeb-f64eb32e595a\") "
Jul 28 15:27:12.733389 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.733244    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/000c0b01-1123-4774-9aeb-f64eb32e595a-openshift-service-ca\") pod \"000c0b01-1123-4774-9aeb-f64eb32e595a\" (UID: \"000c0b01-1123-4774-9aeb-f64eb32e595a\") "
Jul 28 15:27:12.733389 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.733287    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tflvr\" (UniqueName: \"kubernetes.io/projected/000c0b01-1123-4774-9aeb-f64eb32e595a-kube-api-access-tflvr\") pod \"000c0b01-1123-4774-9aeb-f64eb32e595a\" (UID: \"000c0b01-1123-4774-9aeb-f64eb32e595a\") "
Jul 28 15:27:12.733916 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.733882    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/000c0b01-1123-4774-9aeb-f64eb32e595a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "000c0b01-1123-4774-9aeb-f64eb32e595a" (UID: "000c0b01-1123-4774-9aeb-f64eb32e595a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:27:12.737410 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.737380    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/000c0b01-1123-4774-9aeb-f64eb32e595a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "000c0b01-1123-4774-9aeb-f64eb32e595a" (UID: "000c0b01-1123-4774-9aeb-f64eb32e595a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:27:12.737410 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.737403    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/000c0b01-1123-4774-9aeb-f64eb32e595a-kube-api-access-tflvr" (OuterVolumeSpecName: "kube-api-access-tflvr") pod "000c0b01-1123-4774-9aeb-f64eb32e595a" (UID: "000c0b01-1123-4774-9aeb-f64eb32e595a"). InnerVolumeSpecName "kube-api-access-tflvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:27:12.772865 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.772843    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt"
Jul 28 15:27:12.834376 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.834345    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vb5hd\" (UniqueName: \"kubernetes.io/projected/279bbc3f-03c7-4f6f-a30b-daf6940e7c92-kube-api-access-vb5hd\") pod \"279bbc3f-03c7-4f6f-a30b-daf6940e7c92\" (UID: \"279bbc3f-03c7-4f6f-a30b-daf6940e7c92\") "
Jul 28 15:27:12.834521 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.834410    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/279bbc3f-03c7-4f6f-a30b-daf6940e7c92-plugins-config-volume\") pod \"279bbc3f-03c7-4f6f-a30b-daf6940e7c92\" (UID: \"279bbc3f-03c7-4f6f-a30b-daf6940e7c92\") "
Jul 28 15:27:12.834680 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.834668    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/000c0b01-1123-4774-9aeb-f64eb32e595a-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:12.834725 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.834684    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/000c0b01-1123-4774-9aeb-f64eb32e595a-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:12.834725 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.834693    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tflvr\" (UniqueName: \"kubernetes.io/projected/000c0b01-1123-4774-9aeb-f64eb32e595a-kube-api-access-tflvr\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:12.834847 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.834820    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/279bbc3f-03c7-4f6f-a30b-daf6940e7c92-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "279bbc3f-03c7-4f6f-a30b-daf6940e7c92" (UID: "279bbc3f-03c7-4f6f-a30b-daf6940e7c92"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:27:12.836422 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.836404    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/279bbc3f-03c7-4f6f-a30b-daf6940e7c92-kube-api-access-vb5hd" (OuterVolumeSpecName: "kube-api-access-vb5hd") pod "279bbc3f-03c7-4f6f-a30b-daf6940e7c92" (UID: "279bbc3f-03c7-4f6f-a30b-daf6940e7c92"). InnerVolumeSpecName "kube-api-access-vb5hd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:27:12.935621 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.935535    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vb5hd\" (UniqueName: \"kubernetes.io/projected/279bbc3f-03c7-4f6f-a30b-daf6940e7c92-kube-api-access-vb5hd\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:12.935621 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:12.935566    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/279bbc3f-03c7-4f6f-a30b-daf6940e7c92-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:13.496950 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:13.496913    2584 generic.go:356] "Generic (PLEG): container finished" podID="000c0b01-1123-4774-9aeb-f64eb32e595a" containerID="15782a683b95248926d5c20acf22dd31f009807ed1e5be0e771283345a81b3b0" exitCode=0
Jul 28 15:27:13.497420 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:13.496964    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"
Jul 28 15:27:13.497420 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:13.496999    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k" event={"ID":"000c0b01-1123-4774-9aeb-f64eb32e595a","Type":"ContainerDied","Data":"15782a683b95248926d5c20acf22dd31f009807ed1e5be0e771283345a81b3b0"}
Jul 28 15:27:13.497420 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:13.497056    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k" event={"ID":"000c0b01-1123-4774-9aeb-f64eb32e595a","Type":"ContainerDied","Data":"e21cd559d3f068f95632cae4180c7b812e898a6fea31b9a11735c85e8ff7ac54"}
Jul 28 15:27:13.497420 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:13.497084    2584 scope.go:117] "RemoveContainer" containerID="15782a683b95248926d5c20acf22dd31f009807ed1e5be0e771283345a81b3b0"
Jul 28 15:27:13.498650 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:13.498550    2584 generic.go:356] "Generic (PLEG): container finished" podID="279bbc3f-03c7-4f6f-a30b-daf6940e7c92" containerID="2919fcda8f38279525cc9e7803562e1cc6f78dcc2fd24aadcc7f4613c491ca34" exitCode=0
Jul 28 15:27:13.498650 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:13.498601    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt"
Jul 28 15:27:13.498792 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:13.498594    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt" event={"ID":"279bbc3f-03c7-4f6f-a30b-daf6940e7c92","Type":"ContainerDied","Data":"2919fcda8f38279525cc9e7803562e1cc6f78dcc2fd24aadcc7f4613c491ca34"}
Jul 28 15:27:13.498792 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:13.498732    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt" event={"ID":"279bbc3f-03c7-4f6f-a30b-daf6940e7c92","Type":"ContainerDied","Data":"ed6880e94d6f27f2983c50ded279e64d573db18bb3343803e3894501533ec402"}
Jul 28 15:27:13.506330 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:13.506305    2584 scope.go:117] "RemoveContainer" containerID="15782a683b95248926d5c20acf22dd31f009807ed1e5be0e771283345a81b3b0"
Jul 28 15:27:13.506637 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:27:13.506580    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15782a683b95248926d5c20acf22dd31f009807ed1e5be0e771283345a81b3b0\": container with ID starting with 15782a683b95248926d5c20acf22dd31f009807ed1e5be0e771283345a81b3b0 not found: ID does not exist" containerID="15782a683b95248926d5c20acf22dd31f009807ed1e5be0e771283345a81b3b0"
Jul 28 15:27:13.506637 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:13.506609    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15782a683b95248926d5c20acf22dd31f009807ed1e5be0e771283345a81b3b0"} err="failed to get container status \"15782a683b95248926d5c20acf22dd31f009807ed1e5be0e771283345a81b3b0\": rpc error: code = NotFound desc = could not find container \"15782a683b95248926d5c20acf22dd31f009807ed1e5be0e771283345a81b3b0\": container with ID starting with 15782a683b95248926d5c20acf22dd31f009807ed1e5be0e771283345a81b3b0 not found: ID does not exist"
Jul 28 15:27:13.506637 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:13.506632    2584 scope.go:117] "RemoveContainer" containerID="2919fcda8f38279525cc9e7803562e1cc6f78dcc2fd24aadcc7f4613c491ca34"
Jul 28 15:27:13.515044 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:13.515025    2584 scope.go:117] "RemoveContainer" containerID="2919fcda8f38279525cc9e7803562e1cc6f78dcc2fd24aadcc7f4613c491ca34"
Jul 28 15:27:13.515341 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:27:13.515314    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2919fcda8f38279525cc9e7803562e1cc6f78dcc2fd24aadcc7f4613c491ca34\": container with ID starting with 2919fcda8f38279525cc9e7803562e1cc6f78dcc2fd24aadcc7f4613c491ca34 not found: ID does not exist" containerID="2919fcda8f38279525cc9e7803562e1cc6f78dcc2fd24aadcc7f4613c491ca34"
Jul 28 15:27:13.515404 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:13.515343    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2919fcda8f38279525cc9e7803562e1cc6f78dcc2fd24aadcc7f4613c491ca34"} err="failed to get container status \"2919fcda8f38279525cc9e7803562e1cc6f78dcc2fd24aadcc7f4613c491ca34\": rpc error: code = NotFound desc = could not find container \"2919fcda8f38279525cc9e7803562e1cc6f78dcc2fd24aadcc7f4613c491ca34\": container with ID starting with 2919fcda8f38279525cc9e7803562e1cc6f78dcc2fd24aadcc7f4613c491ca34 not found: ID does not exist"
Jul 28 15:27:13.524447 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:13.524423    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"]
Jul 28 15:27:13.528755 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:13.528733    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-034645-6555d6b5cb-rdz7k"]
Jul 28 15:27:13.541796 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:13.541641    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt"]
Jul 28 15:27:13.543654 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:13.543632    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-034645-7fc747d676-v86rt"]
Jul 28 15:27:14.797085 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:14.797050    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="000c0b01-1123-4774-9aeb-f64eb32e595a" path="/var/lib/kubelet/pods/000c0b01-1123-4774-9aeb-f64eb32e595a/volumes"
Jul 28 15:27:14.797481 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:14.797439    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="279bbc3f-03c7-4f6f-a30b-daf6940e7c92" path="/var/lib/kubelet/pods/279bbc3f-03c7-4f6f-a30b-daf6940e7c92/volumes"
Jul 28 15:27:28.675868 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:28.675833    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"]
Jul 28 15:27:28.676359 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:28.676130    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc" podUID="33658123-a2fc-4f67-bb97-03711502555c" containerName="maas-api" containerID="cri-o://a02e05e0261983f65e382b4a282ad7a1bd7d4530b330da9189ad7242045a8884" gracePeriod=30
Jul 28 15:27:28.754826 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:28.754789    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw"]
Jul 28 15:27:28.755069 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:28.755042    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw" podUID="9e0bfb43-4a16-4739-9bef-b0ec4d4bb164" containerName="payload-processing" containerID="cri-o://a5f06b2f68dd7a3c0dae5334acd2b68ec5c2bc7dd3d5b06df8c14b81dc9511ba" gracePeriod=30
Jul 28 15:27:28.760496 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:28.760465    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j"]
Jul 28 15:27:28.760726 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:28.760695    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j" podUID="344c75f5-e576-4598-a2c9-1ecfe0a7f7c6" containerName="payload-pre-processing" containerID="cri-o://16ef231ded6de35697512e6361926421f9e39832b368ff512bf51923365d1cdb" gracePeriod=30
Jul 28 15:27:29.080369 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.080338    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw"
Jul 28 15:27:29.083754 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.083728    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"
Jul 28 15:27:29.087416 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.087392    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j"
Jul 28 15:27:29.200175 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.200037    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e0bfb43-4a16-4739-9bef-b0ec4d4bb164-plugins-config-volume\") pod \"9e0bfb43-4a16-4739-9bef-b0ec4d4bb164\" (UID: \"9e0bfb43-4a16-4739-9bef-b0ec4d4bb164\") "
Jul 28 15:27:29.200175 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.200139    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33658123-a2fc-4f67-bb97-03711502555c-openshift-service-ca\") pod \"33658123-a2fc-4f67-bb97-03711502555c\" (UID: \"33658123-a2fc-4f67-bb97-03711502555c\") "
Jul 28 15:27:29.200415 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.200203    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33658123-a2fc-4f67-bb97-03711502555c-maas-api-tls\") pod \"33658123-a2fc-4f67-bb97-03711502555c\" (UID: \"33658123-a2fc-4f67-bb97-03711502555c\") "
Jul 28 15:27:29.200415 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.200232    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxjrc\" (UniqueName: \"kubernetes.io/projected/33658123-a2fc-4f67-bb97-03711502555c-kube-api-access-kxjrc\") pod \"33658123-a2fc-4f67-bb97-03711502555c\" (UID: \"33658123-a2fc-4f67-bb97-03711502555c\") "
Jul 28 15:27:29.200415 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.200256    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5jhw\" (UniqueName: \"kubernetes.io/projected/9e0bfb43-4a16-4739-9bef-b0ec4d4bb164-kube-api-access-s5jhw\") pod \"9e0bfb43-4a16-4739-9bef-b0ec4d4bb164\" (UID: \"9e0bfb43-4a16-4739-9bef-b0ec4d4bb164\") "
Jul 28 15:27:29.200415 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.200317    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kj8zr\" (UniqueName: \"kubernetes.io/projected/344c75f5-e576-4598-a2c9-1ecfe0a7f7c6-kube-api-access-kj8zr\") pod \"344c75f5-e576-4598-a2c9-1ecfe0a7f7c6\" (UID: \"344c75f5-e576-4598-a2c9-1ecfe0a7f7c6\") "
Jul 28 15:27:29.200415 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.200373    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/344c75f5-e576-4598-a2c9-1ecfe0a7f7c6-plugins-config-volume\") pod \"344c75f5-e576-4598-a2c9-1ecfe0a7f7c6\" (UID: \"344c75f5-e576-4598-a2c9-1ecfe0a7f7c6\") "
Jul 28 15:27:29.200670 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.200550    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e0bfb43-4a16-4739-9bef-b0ec4d4bb164-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9e0bfb43-4a16-4739-9bef-b0ec4d4bb164" (UID: "9e0bfb43-4a16-4739-9bef-b0ec4d4bb164"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:27:29.200670 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.200622    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33658123-a2fc-4f67-bb97-03711502555c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "33658123-a2fc-4f67-bb97-03711502555c" (UID: "33658123-a2fc-4f67-bb97-03711502555c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:27:29.200895 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.200864    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/344c75f5-e576-4598-a2c9-1ecfe0a7f7c6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "344c75f5-e576-4598-a2c9-1ecfe0a7f7c6" (UID: "344c75f5-e576-4598-a2c9-1ecfe0a7f7c6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:27:29.200967 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.200899    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e0bfb43-4a16-4739-9bef-b0ec4d4bb164-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:29.200967 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.200922    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33658123-a2fc-4f67-bb97-03711502555c-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:29.202578 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.202548    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33658123-a2fc-4f67-bb97-03711502555c-kube-api-access-kxjrc" (OuterVolumeSpecName: "kube-api-access-kxjrc") pod "33658123-a2fc-4f67-bb97-03711502555c" (UID: "33658123-a2fc-4f67-bb97-03711502555c"). InnerVolumeSpecName "kube-api-access-kxjrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:27:29.202578 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.202559    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/344c75f5-e576-4598-a2c9-1ecfe0a7f7c6-kube-api-access-kj8zr" (OuterVolumeSpecName: "kube-api-access-kj8zr") pod "344c75f5-e576-4598-a2c9-1ecfe0a7f7c6" (UID: "344c75f5-e576-4598-a2c9-1ecfe0a7f7c6"). InnerVolumeSpecName "kube-api-access-kj8zr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:27:29.202837 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.202814    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e0bfb43-4a16-4739-9bef-b0ec4d4bb164-kube-api-access-s5jhw" (OuterVolumeSpecName: "kube-api-access-s5jhw") pod "9e0bfb43-4a16-4739-9bef-b0ec4d4bb164" (UID: "9e0bfb43-4a16-4739-9bef-b0ec4d4bb164"). InnerVolumeSpecName "kube-api-access-s5jhw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:27:29.202916 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.202817    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33658123-a2fc-4f67-bb97-03711502555c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "33658123-a2fc-4f67-bb97-03711502555c" (UID: "33658123-a2fc-4f67-bb97-03711502555c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:27:29.302305 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.302254    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kj8zr\" (UniqueName: \"kubernetes.io/projected/344c75f5-e576-4598-a2c9-1ecfe0a7f7c6-kube-api-access-kj8zr\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:29.302305 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.302297    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/344c75f5-e576-4598-a2c9-1ecfe0a7f7c6-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:29.302305 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.302311    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33658123-a2fc-4f67-bb97-03711502555c-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:29.302573 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.302320    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxjrc\" (UniqueName: \"kubernetes.io/projected/33658123-a2fc-4f67-bb97-03711502555c-kube-api-access-kxjrc\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:29.302573 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.302330    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5jhw\" (UniqueName: \"kubernetes.io/projected/9e0bfb43-4a16-4739-9bef-b0ec4d4bb164-kube-api-access-s5jhw\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:29.586644 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.586557    2584 generic.go:356] "Generic (PLEG): container finished" podID="9e0bfb43-4a16-4739-9bef-b0ec4d4bb164" containerID="a5f06b2f68dd7a3c0dae5334acd2b68ec5c2bc7dd3d5b06df8c14b81dc9511ba" exitCode=0
Jul 28 15:27:29.586644 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.586603    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw"
Jul 28 15:27:29.586644 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.586631    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw" event={"ID":"9e0bfb43-4a16-4739-9bef-b0ec4d4bb164","Type":"ContainerDied","Data":"a5f06b2f68dd7a3c0dae5334acd2b68ec5c2bc7dd3d5b06df8c14b81dc9511ba"}
Jul 28 15:27:29.586899 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.586682    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw" event={"ID":"9e0bfb43-4a16-4739-9bef-b0ec4d4bb164","Type":"ContainerDied","Data":"55bb5e1c5782bb7dd7322507b2210e5dff5604760d48d11bbc0b8e6f696d32e6"}
Jul 28 15:27:29.586899 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.586710    2584 scope.go:117] "RemoveContainer" containerID="a5f06b2f68dd7a3c0dae5334acd2b68ec5c2bc7dd3d5b06df8c14b81dc9511ba"
Jul 28 15:27:29.588015 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.587982    2584 generic.go:356] "Generic (PLEG): container finished" podID="344c75f5-e576-4598-a2c9-1ecfe0a7f7c6" containerID="16ef231ded6de35697512e6361926421f9e39832b368ff512bf51923365d1cdb" exitCode=0
Jul 28 15:27:29.588161 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.588048    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j"
Jul 28 15:27:29.588161 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.588064    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j" event={"ID":"344c75f5-e576-4598-a2c9-1ecfe0a7f7c6","Type":"ContainerDied","Data":"16ef231ded6de35697512e6361926421f9e39832b368ff512bf51923365d1cdb"}
Jul 28 15:27:29.588161 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.588132    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j" event={"ID":"344c75f5-e576-4598-a2c9-1ecfe0a7f7c6","Type":"ContainerDied","Data":"a1852f6d26c777db2ae3bbceae0b7f3424b48fb5829a1ba2fdd40512eb66d934"}
Jul 28 15:27:29.589619 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.589590    2584 generic.go:356] "Generic (PLEG): container finished" podID="33658123-a2fc-4f67-bb97-03711502555c" containerID="a02e05e0261983f65e382b4a282ad7a1bd7d4530b330da9189ad7242045a8884" exitCode=0
Jul 28 15:27:29.589748 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.589639    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"
Jul 28 15:27:29.589748 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.589659    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc" event={"ID":"33658123-a2fc-4f67-bb97-03711502555c","Type":"ContainerDied","Data":"a02e05e0261983f65e382b4a282ad7a1bd7d4530b330da9189ad7242045a8884"}
Jul 28 15:27:29.589748 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.589699    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc" event={"ID":"33658123-a2fc-4f67-bb97-03711502555c","Type":"ContainerDied","Data":"0c77b6829ef740dd928fa415ae5c1ab822a08d52666b43928856b493606138d5"}
Jul 28 15:27:29.596687 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.596666    2584 scope.go:117] "RemoveContainer" containerID="a5f06b2f68dd7a3c0dae5334acd2b68ec5c2bc7dd3d5b06df8c14b81dc9511ba"
Jul 28 15:27:29.597024 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:27:29.596989    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5f06b2f68dd7a3c0dae5334acd2b68ec5c2bc7dd3d5b06df8c14b81dc9511ba\": container with ID starting with a5f06b2f68dd7a3c0dae5334acd2b68ec5c2bc7dd3d5b06df8c14b81dc9511ba not found: ID does not exist" containerID="a5f06b2f68dd7a3c0dae5334acd2b68ec5c2bc7dd3d5b06df8c14b81dc9511ba"
Jul 28 15:27:29.597155 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.597040    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5f06b2f68dd7a3c0dae5334acd2b68ec5c2bc7dd3d5b06df8c14b81dc9511ba"} err="failed to get container status \"a5f06b2f68dd7a3c0dae5334acd2b68ec5c2bc7dd3d5b06df8c14b81dc9511ba\": rpc error: code = NotFound desc = could not find container \"a5f06b2f68dd7a3c0dae5334acd2b68ec5c2bc7dd3d5b06df8c14b81dc9511ba\": container with ID starting with a5f06b2f68dd7a3c0dae5334acd2b68ec5c2bc7dd3d5b06df8c14b81dc9511ba not found: ID does not exist"
Jul 28 15:27:29.597155 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.597067    2584 scope.go:117] "RemoveContainer" containerID="16ef231ded6de35697512e6361926421f9e39832b368ff512bf51923365d1cdb"
Jul 28 15:27:29.605966 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.605946    2584 scope.go:117] "RemoveContainer" containerID="16ef231ded6de35697512e6361926421f9e39832b368ff512bf51923365d1cdb"
Jul 28 15:27:29.606286 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:27:29.606257    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16ef231ded6de35697512e6361926421f9e39832b368ff512bf51923365d1cdb\": container with ID starting with 16ef231ded6de35697512e6361926421f9e39832b368ff512bf51923365d1cdb not found: ID does not exist" containerID="16ef231ded6de35697512e6361926421f9e39832b368ff512bf51923365d1cdb"
Jul 28 15:27:29.606388 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.606294    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16ef231ded6de35697512e6361926421f9e39832b368ff512bf51923365d1cdb"} err="failed to get container status \"16ef231ded6de35697512e6361926421f9e39832b368ff512bf51923365d1cdb\": rpc error: code = NotFound desc = could not find container \"16ef231ded6de35697512e6361926421f9e39832b368ff512bf51923365d1cdb\": container with ID starting with 16ef231ded6de35697512e6361926421f9e39832b368ff512bf51923365d1cdb not found: ID does not exist"
Jul 28 15:27:29.606388 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.606313    2584 scope.go:117] "RemoveContainer" containerID="a02e05e0261983f65e382b4a282ad7a1bd7d4530b330da9189ad7242045a8884"
Jul 28 15:27:29.615992 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.615971    2584 scope.go:117] "RemoveContainer" containerID="a02e05e0261983f65e382b4a282ad7a1bd7d4530b330da9189ad7242045a8884"
Jul 28 15:27:29.616369 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:27:29.616342    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a02e05e0261983f65e382b4a282ad7a1bd7d4530b330da9189ad7242045a8884\": container with ID starting with a02e05e0261983f65e382b4a282ad7a1bd7d4530b330da9189ad7242045a8884 not found: ID does not exist" containerID="a02e05e0261983f65e382b4a282ad7a1bd7d4530b330da9189ad7242045a8884"
Jul 28 15:27:29.616458 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.616381    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a02e05e0261983f65e382b4a282ad7a1bd7d4530b330da9189ad7242045a8884"} err="failed to get container status \"a02e05e0261983f65e382b4a282ad7a1bd7d4530b330da9189ad7242045a8884\": rpc error: code = NotFound desc = could not find container \"a02e05e0261983f65e382b4a282ad7a1bd7d4530b330da9189ad7242045a8884\": container with ID starting with a02e05e0261983f65e382b4a282ad7a1bd7d4530b330da9189ad7242045a8884 not found: ID does not exist"
Jul 28 15:27:29.616538 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.616515    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j"]
Jul 28 15:27:29.619522 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.619501    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d96a19-79885d98c6-v8p6j"]
Jul 28 15:27:29.632049 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.632019    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"]
Jul 28 15:27:29.635065 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.635037    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d96a19-6b657dd759-nkdvc"]
Jul 28 15:27:29.647041 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.647008    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw"]
Jul 28 15:27:29.653530 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:29.653502    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-d96a19-d97c7bd46-gxllw"]
Jul 28 15:27:30.796941 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:30.796901    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33658123-a2fc-4f67-bb97-03711502555c" path="/var/lib/kubelet/pods/33658123-a2fc-4f67-bb97-03711502555c/volumes"
Jul 28 15:27:30.797357 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:30.797288    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="344c75f5-e576-4598-a2c9-1ecfe0a7f7c6" path="/var/lib/kubelet/pods/344c75f5-e576-4598-a2c9-1ecfe0a7f7c6/volumes"
Jul 28 15:27:30.797690 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:30.797666    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e0bfb43-4a16-4739-9bef-b0ec4d4bb164" path="/var/lib/kubelet/pods/9e0bfb43-4a16-4739-9bef-b0ec4d4bb164/volumes"
Jul 28 15:27:44.861000 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:44.860914    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"]
Jul 28 15:27:44.861600 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:44.861207    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h" podUID="a8e46d34-99a8-4ef4-a26d-36783061146b" containerName="maas-api" containerID="cri-o://10bd0d2e6648a00ab7ecaec1e16a4e269c100f1ec34702c6ecafd2c7a7af1eb9" gracePeriod=30
Jul 28 15:27:44.932158 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:44.932122    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542"]
Jul 28 15:27:44.932447 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:44.932386    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542" podUID="c1ab044b-c06b-46cb-b13f-b10885d9b835" containerName="payload-processing" containerID="cri-o://b81e964ba520c5fdc58dbfc1d816427e8da5a9c031a20960a0db81542f899b13" gracePeriod=30
Jul 28 15:27:44.947750 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:44.947161    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d"]
Jul 28 15:27:44.947750 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:44.947393    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d" podUID="eb7245e4-ecbf-465f-8203-54fb9db71ea0" containerName="payload-pre-processing" containerID="cri-o://cea1c4a34a1a6690f97919a683d6d74004b4b84d59c0a8752b3d699e696a64a4" gracePeriod=30
Jul 28 15:27:45.241816 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.241774    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542"
Jul 28 15:27:45.255040 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.255012    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d"
Jul 28 15:27:45.258610 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.258586    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"
Jul 28 15:27:45.348381 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.348350    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1ab044b-c06b-46cb-b13f-b10885d9b835-plugins-config-volume\") pod \"c1ab044b-c06b-46cb-b13f-b10885d9b835\" (UID: \"c1ab044b-c06b-46cb-b13f-b10885d9b835\") "
Jul 28 15:27:45.348646 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.348408    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8e46d34-99a8-4ef4-a26d-36783061146b-openshift-service-ca\") pod \"a8e46d34-99a8-4ef4-a26d-36783061146b\" (UID: \"a8e46d34-99a8-4ef4-a26d-36783061146b\") "
Jul 28 15:27:45.348646 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.348470    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhl5p\" (UniqueName: \"kubernetes.io/projected/c1ab044b-c06b-46cb-b13f-b10885d9b835-kube-api-access-jhl5p\") pod \"c1ab044b-c06b-46cb-b13f-b10885d9b835\" (UID: \"c1ab044b-c06b-46cb-b13f-b10885d9b835\") "
Jul 28 15:27:45.348646 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.348499    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v79j\" (UniqueName: \"kubernetes.io/projected/eb7245e4-ecbf-465f-8203-54fb9db71ea0-kube-api-access-6v79j\") pod \"eb7245e4-ecbf-465f-8203-54fb9db71ea0\" (UID: \"eb7245e4-ecbf-465f-8203-54fb9db71ea0\") "
Jul 28 15:27:45.348646 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.348526    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb7245e4-ecbf-465f-8203-54fb9db71ea0-plugins-config-volume\") pod \"eb7245e4-ecbf-465f-8203-54fb9db71ea0\" (UID: \"eb7245e4-ecbf-465f-8203-54fb9db71ea0\") "
Jul 28 15:27:45.348887 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.348814    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8e46d34-99a8-4ef4-a26d-36783061146b-maas-api-tls\") pod \"a8e46d34-99a8-4ef4-a26d-36783061146b\" (UID: \"a8e46d34-99a8-4ef4-a26d-36783061146b\") "
Jul 28 15:27:45.348887 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.348869    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m4rf\" (UniqueName: \"kubernetes.io/projected/a8e46d34-99a8-4ef4-a26d-36783061146b-kube-api-access-6m4rf\") pod \"a8e46d34-99a8-4ef4-a26d-36783061146b\" (UID: \"a8e46d34-99a8-4ef4-a26d-36783061146b\") "
Jul 28 15:27:45.348887 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.348868    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8e46d34-99a8-4ef4-a26d-36783061146b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a8e46d34-99a8-4ef4-a26d-36783061146b" (UID: "a8e46d34-99a8-4ef4-a26d-36783061146b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:27:45.349248 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.349220    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1ab044b-c06b-46cb-b13f-b10885d9b835-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c1ab044b-c06b-46cb-b13f-b10885d9b835" (UID: "c1ab044b-c06b-46cb-b13f-b10885d9b835"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:27:45.349489 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.349244    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb7245e4-ecbf-465f-8203-54fb9db71ea0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eb7245e4-ecbf-465f-8203-54fb9db71ea0" (UID: "eb7245e4-ecbf-465f-8203-54fb9db71ea0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:27:45.349615 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.349241    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8e46d34-99a8-4ef4-a26d-36783061146b-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:45.351049 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.351021    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8e46d34-99a8-4ef4-a26d-36783061146b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a8e46d34-99a8-4ef4-a26d-36783061146b" (UID: "a8e46d34-99a8-4ef4-a26d-36783061146b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:27:45.351194 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.351172    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1ab044b-c06b-46cb-b13f-b10885d9b835-kube-api-access-jhl5p" (OuterVolumeSpecName: "kube-api-access-jhl5p") pod "c1ab044b-c06b-46cb-b13f-b10885d9b835" (UID: "c1ab044b-c06b-46cb-b13f-b10885d9b835"). InnerVolumeSpecName "kube-api-access-jhl5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:27:45.351269 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.351236    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb7245e4-ecbf-465f-8203-54fb9db71ea0-kube-api-access-6v79j" (OuterVolumeSpecName: "kube-api-access-6v79j") pod "eb7245e4-ecbf-465f-8203-54fb9db71ea0" (UID: "eb7245e4-ecbf-465f-8203-54fb9db71ea0"). InnerVolumeSpecName "kube-api-access-6v79j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:27:45.351331 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.351313    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8e46d34-99a8-4ef4-a26d-36783061146b-kube-api-access-6m4rf" (OuterVolumeSpecName: "kube-api-access-6m4rf") pod "a8e46d34-99a8-4ef4-a26d-36783061146b" (UID: "a8e46d34-99a8-4ef4-a26d-36783061146b"). InnerVolumeSpecName "kube-api-access-6m4rf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:27:45.451032 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.450944    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhl5p\" (UniqueName: \"kubernetes.io/projected/c1ab044b-c06b-46cb-b13f-b10885d9b835-kube-api-access-jhl5p\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:45.451032 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.450973    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v79j\" (UniqueName: \"kubernetes.io/projected/eb7245e4-ecbf-465f-8203-54fb9db71ea0-kube-api-access-6v79j\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:45.451032 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.450983    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb7245e4-ecbf-465f-8203-54fb9db71ea0-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:45.451032 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.450993    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8e46d34-99a8-4ef4-a26d-36783061146b-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:45.451032 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.451002    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m4rf\" (UniqueName: \"kubernetes.io/projected/a8e46d34-99a8-4ef4-a26d-36783061146b-kube-api-access-6m4rf\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:45.451032 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.451009    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1ab044b-c06b-46cb-b13f-b10885d9b835-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:45.679281 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.679239    2584 generic.go:356] "Generic (PLEG): container finished" podID="a8e46d34-99a8-4ef4-a26d-36783061146b" containerID="10bd0d2e6648a00ab7ecaec1e16a4e269c100f1ec34702c6ecafd2c7a7af1eb9" exitCode=0
Jul 28 15:27:45.679529 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.679290    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"
Jul 28 15:27:45.679529 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.679324    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h" event={"ID":"a8e46d34-99a8-4ef4-a26d-36783061146b","Type":"ContainerDied","Data":"10bd0d2e6648a00ab7ecaec1e16a4e269c100f1ec34702c6ecafd2c7a7af1eb9"}
Jul 28 15:27:45.679529 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.679388    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h" event={"ID":"a8e46d34-99a8-4ef4-a26d-36783061146b","Type":"ContainerDied","Data":"24f1fa6f24a3427eeb09cf0b80d2cae3cabcbacb66542ea730b54ee712d0cb26"}
Jul 28 15:27:45.679529 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.679419    2584 scope.go:117] "RemoveContainer" containerID="10bd0d2e6648a00ab7ecaec1e16a4e269c100f1ec34702c6ecafd2c7a7af1eb9"
Jul 28 15:27:45.680815 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.680783    2584 generic.go:356] "Generic (PLEG): container finished" podID="eb7245e4-ecbf-465f-8203-54fb9db71ea0" containerID="cea1c4a34a1a6690f97919a683d6d74004b4b84d59c0a8752b3d699e696a64a4" exitCode=0
Jul 28 15:27:45.680930 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.680841    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d"
Jul 28 15:27:45.680930 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.680845    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d" event={"ID":"eb7245e4-ecbf-465f-8203-54fb9db71ea0","Type":"ContainerDied","Data":"cea1c4a34a1a6690f97919a683d6d74004b4b84d59c0a8752b3d699e696a64a4"}
Jul 28 15:27:45.680930 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.680893    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d" event={"ID":"eb7245e4-ecbf-465f-8203-54fb9db71ea0","Type":"ContainerDied","Data":"22159b32b498fb90f543855516704404ce0af086c0608375afac97275fe061d5"}
Jul 28 15:27:45.682347 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.682321    2584 generic.go:356] "Generic (PLEG): container finished" podID="c1ab044b-c06b-46cb-b13f-b10885d9b835" containerID="b81e964ba520c5fdc58dbfc1d816427e8da5a9c031a20960a0db81542f899b13" exitCode=0
Jul 28 15:27:45.682487 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.682362    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542"
Jul 28 15:27:45.682487 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.682412    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542" event={"ID":"c1ab044b-c06b-46cb-b13f-b10885d9b835","Type":"ContainerDied","Data":"b81e964ba520c5fdc58dbfc1d816427e8da5a9c031a20960a0db81542f899b13"}
Jul 28 15:27:45.682487 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.682451    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542" event={"ID":"c1ab044b-c06b-46cb-b13f-b10885d9b835","Type":"ContainerDied","Data":"ebc747564ebf67222caa4aaf275dec0ca9cfa5cdc15f07fc66e38ef944e701b5"}
Jul 28 15:27:45.691051 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.691027    2584 scope.go:117] "RemoveContainer" containerID="10bd0d2e6648a00ab7ecaec1e16a4e269c100f1ec34702c6ecafd2c7a7af1eb9"
Jul 28 15:27:45.691390 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:27:45.691368    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10bd0d2e6648a00ab7ecaec1e16a4e269c100f1ec34702c6ecafd2c7a7af1eb9\": container with ID starting with 10bd0d2e6648a00ab7ecaec1e16a4e269c100f1ec34702c6ecafd2c7a7af1eb9 not found: ID does not exist" containerID="10bd0d2e6648a00ab7ecaec1e16a4e269c100f1ec34702c6ecafd2c7a7af1eb9"
Jul 28 15:27:45.691500 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.691402    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10bd0d2e6648a00ab7ecaec1e16a4e269c100f1ec34702c6ecafd2c7a7af1eb9"} err="failed to get container status \"10bd0d2e6648a00ab7ecaec1e16a4e269c100f1ec34702c6ecafd2c7a7af1eb9\": rpc error: code = NotFound desc = could not find container \"10bd0d2e6648a00ab7ecaec1e16a4e269c100f1ec34702c6ecafd2c7a7af1eb9\": container with ID starting with 10bd0d2e6648a00ab7ecaec1e16a4e269c100f1ec34702c6ecafd2c7a7af1eb9 not found: ID does not exist"
Jul 28 15:27:45.691500 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.691429    2584 scope.go:117] "RemoveContainer" containerID="cea1c4a34a1a6690f97919a683d6d74004b4b84d59c0a8752b3d699e696a64a4"
Jul 28 15:27:45.700079 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.700057    2584 scope.go:117] "RemoveContainer" containerID="cea1c4a34a1a6690f97919a683d6d74004b4b84d59c0a8752b3d699e696a64a4"
Jul 28 15:27:45.700383 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:27:45.700366    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cea1c4a34a1a6690f97919a683d6d74004b4b84d59c0a8752b3d699e696a64a4\": container with ID starting with cea1c4a34a1a6690f97919a683d6d74004b4b84d59c0a8752b3d699e696a64a4 not found: ID does not exist" containerID="cea1c4a34a1a6690f97919a683d6d74004b4b84d59c0a8752b3d699e696a64a4"
Jul 28 15:27:45.700431 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.700389    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea1c4a34a1a6690f97919a683d6d74004b4b84d59c0a8752b3d699e696a64a4"} err="failed to get container status \"cea1c4a34a1a6690f97919a683d6d74004b4b84d59c0a8752b3d699e696a64a4\": rpc error: code = NotFound desc = could not find container \"cea1c4a34a1a6690f97919a683d6d74004b4b84d59c0a8752b3d699e696a64a4\": container with ID starting with cea1c4a34a1a6690f97919a683d6d74004b4b84d59c0a8752b3d699e696a64a4 not found: ID does not exist"
Jul 28 15:27:45.700431 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.700404    2584 scope.go:117] "RemoveContainer" containerID="b81e964ba520c5fdc58dbfc1d816427e8da5a9c031a20960a0db81542f899b13"
Jul 28 15:27:45.708924 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.708900    2584 scope.go:117] "RemoveContainer" containerID="b81e964ba520c5fdc58dbfc1d816427e8da5a9c031a20960a0db81542f899b13"
Jul 28 15:27:45.709333 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:27:45.709306    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b81e964ba520c5fdc58dbfc1d816427e8da5a9c031a20960a0db81542f899b13\": container with ID starting with b81e964ba520c5fdc58dbfc1d816427e8da5a9c031a20960a0db81542f899b13 not found: ID does not exist" containerID="b81e964ba520c5fdc58dbfc1d816427e8da5a9c031a20960a0db81542f899b13"
Jul 28 15:27:45.709415 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.709342    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b81e964ba520c5fdc58dbfc1d816427e8da5a9c031a20960a0db81542f899b13"} err="failed to get container status \"b81e964ba520c5fdc58dbfc1d816427e8da5a9c031a20960a0db81542f899b13\": rpc error: code = NotFound desc = could not find container \"b81e964ba520c5fdc58dbfc1d816427e8da5a9c031a20960a0db81542f899b13\": container with ID starting with b81e964ba520c5fdc58dbfc1d816427e8da5a9c031a20960a0db81542f899b13 not found: ID does not exist"
Jul 28 15:27:45.709525 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.709509    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"]
Jul 28 15:27:45.715231 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.715203    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h"]
Jul 28 15:27:45.731359 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.731326    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542"]
Jul 28 15:27:45.735085 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.735060    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-3edd55-5f6956595d-xk542"]
Jul 28 15:27:45.749508 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.749479    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d"]
Jul 28 15:27:45.751130 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:45.751110    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-3edd55-6b7fc479dd-5c49d"]
Jul 28 15:27:46.797002 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:46.796962    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8e46d34-99a8-4ef4-a26d-36783061146b" path="/var/lib/kubelet/pods/a8e46d34-99a8-4ef4-a26d-36783061146b/volumes"
Jul 28 15:27:46.797431 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:46.797375    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1ab044b-c06b-46cb-b13f-b10885d9b835" path="/var/lib/kubelet/pods/c1ab044b-c06b-46cb-b13f-b10885d9b835/volumes"
Jul 28 15:27:46.797690 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:46.797677    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb7245e4-ecbf-465f-8203-54fb9db71ea0" path="/var/lib/kubelet/pods/eb7245e4-ecbf-465f-8203-54fb9db71ea0/volumes"
Jul 28 15:27:48.164161 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:48.164087    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3edd55-c864c8546-mhn6h" podUID="a8e46d34-99a8-4ef4-a26d-36783061146b" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.83:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 28 15:27:53.084268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:53.084224    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"]
Jul 28 15:27:53.084678 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:53.084455    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq" podUID="c80aaa6e-877c-4437-9b94-72105d055102" containerName="istio-proxy" containerID="cri-o://a3ab0b0cb35a60943b4243ba3f95c14ac5fa6350f5e066a8d1f2eac913ff9237" gracePeriod=30
Jul 28 15:27:53.491732 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:53.491696    2584 patch_prober.go:28] interesting pod/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 15:27:53.491905 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:53.491759    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq" podUID="c80aaa6e-877c-4437-9b94-72105d055102" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 15:27:58.339601 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.339573    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:27:58.475134 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.475012    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-istio-data\") pod \"c80aaa6e-877c-4437-9b94-72105d055102\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") "
Jul 28 15:27:58.475134 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.475067    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c80aaa6e-877c-4437-9b94-72105d055102-istiod-ca-cert\") pod \"c80aaa6e-877c-4437-9b94-72105d055102\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") "
Jul 28 15:27:58.475134 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.475136    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c80aaa6e-877c-4437-9b94-72105d055102-istio-token\") pod \"c80aaa6e-877c-4437-9b94-72105d055102\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") "
Jul 28 15:27:58.475424 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.475164    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfqbf\" (UniqueName: \"kubernetes.io/projected/c80aaa6e-877c-4437-9b94-72105d055102-kube-api-access-pfqbf\") pod \"c80aaa6e-877c-4437-9b94-72105d055102\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") "
Jul 28 15:27:58.475424 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.475192    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-workload-certs\") pod \"c80aaa6e-877c-4437-9b94-72105d055102\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") "
Jul 28 15:27:58.475424 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.475222    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c80aaa6e-877c-4437-9b94-72105d055102-istio-podinfo\") pod \"c80aaa6e-877c-4437-9b94-72105d055102\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") "
Jul 28 15:27:58.475424 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.475282    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-workload-socket\") pod \"c80aaa6e-877c-4437-9b94-72105d055102\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") "
Jul 28 15:27:58.475424 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.475305    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-credential-socket\") pod \"c80aaa6e-877c-4437-9b94-72105d055102\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") "
Jul 28 15:27:58.475424 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.475343    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-istio-envoy\") pod \"c80aaa6e-877c-4437-9b94-72105d055102\" (UID: \"c80aaa6e-877c-4437-9b94-72105d055102\") "
Jul 28 15:27:58.475740 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.475545    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c80aaa6e-877c-4437-9b94-72105d055102" (UID: "c80aaa6e-877c-4437-9b94-72105d055102"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:27:58.475740 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.475565    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c80aaa6e-877c-4437-9b94-72105d055102-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c80aaa6e-877c-4437-9b94-72105d055102" (UID: "c80aaa6e-877c-4437-9b94-72105d055102"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:27:58.475740 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.475687    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c80aaa6e-877c-4437-9b94-72105d055102" (UID: "c80aaa6e-877c-4437-9b94-72105d055102"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:27:58.475740 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.475720    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c80aaa6e-877c-4437-9b94-72105d055102-istiod-ca-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:58.475950 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.475744    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-workload-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:58.475950 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.475843    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c80aaa6e-877c-4437-9b94-72105d055102" (UID: "c80aaa6e-877c-4437-9b94-72105d055102"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:27:58.476074 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.476047    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-istio-data" (OuterVolumeSpecName: "istio-data") pod "c80aaa6e-877c-4437-9b94-72105d055102" (UID: "c80aaa6e-877c-4437-9b94-72105d055102"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:27:58.477458 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.477432    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c80aaa6e-877c-4437-9b94-72105d055102-istio-token" (OuterVolumeSpecName: "istio-token") pod "c80aaa6e-877c-4437-9b94-72105d055102" (UID: "c80aaa6e-877c-4437-9b94-72105d055102"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:27:58.477575 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.477508    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c80aaa6e-877c-4437-9b94-72105d055102" (UID: "c80aaa6e-877c-4437-9b94-72105d055102"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:27:58.477794 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.477774    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c80aaa6e-877c-4437-9b94-72105d055102-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c80aaa6e-877c-4437-9b94-72105d055102" (UID: "c80aaa6e-877c-4437-9b94-72105d055102"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 15:27:58.477864 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.477777    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c80aaa6e-877c-4437-9b94-72105d055102-kube-api-access-pfqbf" (OuterVolumeSpecName: "kube-api-access-pfqbf") pod "c80aaa6e-877c-4437-9b94-72105d055102" (UID: "c80aaa6e-877c-4437-9b94-72105d055102"). InnerVolumeSpecName "kube-api-access-pfqbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:27:58.576769 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.576728    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-istio-data\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:58.576769 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.576762    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c80aaa6e-877c-4437-9b94-72105d055102-istio-token\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:58.576769 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.576772    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfqbf\" (UniqueName: \"kubernetes.io/projected/c80aaa6e-877c-4437-9b94-72105d055102-kube-api-access-pfqbf\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:58.576769 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.576784    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c80aaa6e-877c-4437-9b94-72105d055102-istio-podinfo\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:58.577130 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.576792    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-workload-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:58.577130 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.576800    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-credential-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:58.577130 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.576808    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c80aaa6e-877c-4437-9b94-72105d055102-istio-envoy\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 28 15:27:58.750658 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.750547    2584 generic.go:356] "Generic (PLEG): container finished" podID="c80aaa6e-877c-4437-9b94-72105d055102" containerID="a3ab0b0cb35a60943b4243ba3f95c14ac5fa6350f5e066a8d1f2eac913ff9237" exitCode=0
Jul 28 15:27:58.750658 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.750599    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"
Jul 28 15:27:58.750658 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.750627    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq" event={"ID":"c80aaa6e-877c-4437-9b94-72105d055102","Type":"ContainerDied","Data":"a3ab0b0cb35a60943b4243ba3f95c14ac5fa6350f5e066a8d1f2eac913ff9237"}
Jul 28 15:27:58.750889 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.750686    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq" event={"ID":"c80aaa6e-877c-4437-9b94-72105d055102","Type":"ContainerDied","Data":"f1a99385797b3bdc1a6ed6ee385ca127f3fb66e016758035088f8c208a85102d"}
Jul 28 15:27:58.750889 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.750711    2584 scope.go:117] "RemoveContainer" containerID="a3ab0b0cb35a60943b4243ba3f95c14ac5fa6350f5e066a8d1f2eac913ff9237"
Jul 28 15:27:58.760152 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.760127    2584 scope.go:117] "RemoveContainer" containerID="cbecb1f0bd2e4336749f3152cd2c4ef9007af690f173b9226e41d3f260600840"
Jul 28 15:27:58.768896 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.768876    2584 scope.go:117] "RemoveContainer" containerID="a3ab0b0cb35a60943b4243ba3f95c14ac5fa6350f5e066a8d1f2eac913ff9237"
Jul 28 15:27:58.769193 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:27:58.769172    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3ab0b0cb35a60943b4243ba3f95c14ac5fa6350f5e066a8d1f2eac913ff9237\": container with ID starting with a3ab0b0cb35a60943b4243ba3f95c14ac5fa6350f5e066a8d1f2eac913ff9237 not found: ID does not exist" containerID="a3ab0b0cb35a60943b4243ba3f95c14ac5fa6350f5e066a8d1f2eac913ff9237"
Jul 28 15:27:58.769245 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.769203    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3ab0b0cb35a60943b4243ba3f95c14ac5fa6350f5e066a8d1f2eac913ff9237"} err="failed to get container status \"a3ab0b0cb35a60943b4243ba3f95c14ac5fa6350f5e066a8d1f2eac913ff9237\": rpc error: code = NotFound desc = could not find container \"a3ab0b0cb35a60943b4243ba3f95c14ac5fa6350f5e066a8d1f2eac913ff9237\": container with ID starting with a3ab0b0cb35a60943b4243ba3f95c14ac5fa6350f5e066a8d1f2eac913ff9237 not found: ID does not exist"
Jul 28 15:27:58.769245 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.769223    2584 scope.go:117] "RemoveContainer" containerID="cbecb1f0bd2e4336749f3152cd2c4ef9007af690f173b9226e41d3f260600840"
Jul 28 15:27:58.769506 ip-10-0-130-66 kubenswrapper[2584]: E0728 15:27:58.769487    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbecb1f0bd2e4336749f3152cd2c4ef9007af690f173b9226e41d3f260600840\": container with ID starting with cbecb1f0bd2e4336749f3152cd2c4ef9007af690f173b9226e41d3f260600840 not found: ID does not exist" containerID="cbecb1f0bd2e4336749f3152cd2c4ef9007af690f173b9226e41d3f260600840"
Jul 28 15:27:58.769560 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.769511    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbecb1f0bd2e4336749f3152cd2c4ef9007af690f173b9226e41d3f260600840"} err="failed to get container status \"cbecb1f0bd2e4336749f3152cd2c4ef9007af690f173b9226e41d3f260600840\": rpc error: code = NotFound desc = could not find container \"cbecb1f0bd2e4336749f3152cd2c4ef9007af690f173b9226e41d3f260600840\": container with ID starting with cbecb1f0bd2e4336749f3152cd2c4ef9007af690f173b9226e41d3f260600840 not found: ID does not exist"
Jul 28 15:27:58.788568 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.788522    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"]
Jul 28 15:27:58.797386 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:58.797355    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-3edd55-openshift-default-8567f87944-mpzwq"]
Jul 28 15:27:59.788185 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:59.788156    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-65db4695c7-z9h72_ddf405d5-caef-4e8f-8882-3d5a12a9d41e/manager/0.log"
Jul 28 15:27:59.909536 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:27:59.909504    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dbm5c_14c17f23-5dac-413e-9df4-bf19cfa8c4ef/server/0.log"
Jul 28 15:28:00.036732 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:00.036691    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-fwf8p_e360876b-132e-4a38-bbeb-d6c87c9b8548/manager/0.log"
Jul 28 15:28:00.159157 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:00.159122    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65f7956c87-29zbg_80778efc-8a53-41a7-b6b3-dd27455c0004/copy-manifests/0.log"
Jul 28 15:28:00.173813 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:00.173782    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65f7956c87-29zbg_80778efc-8a53-41a7-b6b3-dd27455c0004/manager/0.log"
Jul 28 15:28:00.797487 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:00.797452    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c80aaa6e-877c-4437-9b94-72105d055102" path="/var/lib/kubelet/pods/c80aaa6e-877c-4437-9b94-72105d055102/volumes"
Jul 28 15:28:00.808042 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:00.808005    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5f4898b989-m22wz_c914ebc9-70b8-46c3-a048-6d4a8fdfb754/maas-api/0.log"
Jul 28 15:28:01.288335 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:01.288293    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-x86gq_ea3e46f5-4237-4fa5-9d99-cc58d394a04b/postgres/0.log"
Jul 28 15:28:03.142240 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:03.142210    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hgs74_7922ad58-46bf-4c59-9713-162ae3ec1bc7/registry-server/0.log"
Jul 28 15:28:03.901866 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:03.901834    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr_a82f1155-e65d-46c1-aa7a-739022d05fc5/istio-proxy/0.log"
Jul 28 15:28:04.031080 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:04.031044    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-4ndsh_e8929d68-d91e-4163-bbfc-f848bba1f652/discovery/0.log"
Jul 28 15:28:04.262054 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:04.261973    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c4fff746-6ksdx_4e108199-9380-42ba-8d02-8361be3b37ad/kube-auth-proxy/0.log"
Jul 28 15:28:04.381018 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:04.380988    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c747fcbf8-4nzkp_0341cb69-6aaa-47f2-8009-28908c0830c7/istio-proxy/1.log"
Jul 28 15:28:04.516874 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:04.516795    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-82vf5_9566b9fe-f896-4d0a-a68a-823856c7aa06/payload-pre-processing/0.log"
Jul 28 15:28:04.646076 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:04.646045    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-2hwhh_5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2/payload-processing/0.log"
Jul 28 15:28:05.147649 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:05.147619    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-xtshk_32e822e1-48a9-4b1e-9404-fc4f3571de8d/main/0.log"
Jul 28 15:28:05.263036 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:05.263002    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-24bm6_a481bb07-b3a5-4a57-ada0-788e5912e506/main/0.log"
Jul 28 15:28:05.622104 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:05.622070    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-p86xh_a5ce7fab-53de-49d9-80bb-66d7fe5843ad/main/0.log"
Jul 28 15:28:14.759742 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:14.759710    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-d7fhl_9ffc9cda-254d-4825-bb1a-1872331a8cc9/global-pull-secret-syncer/0.log"
Jul 28 15:28:15.026428 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:15.026328    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-h2ddz_d943e044-62ab-4f2c-a9a6-10fc30a01aca/konnectivity-agent/0.log"
Jul 28 15:28:15.071573 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:15.071546    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-66.ec2.internal_f675f063811c81df70fa063df9f70b5b/haproxy/0.log"
Jul 28 15:28:21.638022 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:21.637983    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hgs74_7922ad58-46bf-4c59-9713-162ae3ec1bc7/registry-server/0.log"
Jul 28 15:28:23.537404 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537364    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"]
Jul 28 15:28:23.537808 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537722    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c80aaa6e-877c-4437-9b94-72105d055102" containerName="istio-proxy"
Jul 28 15:28:23.537808 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537732    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c80aaa6e-877c-4437-9b94-72105d055102" containerName="istio-proxy"
Jul 28 15:28:23.537808 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537741    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e0bfb43-4a16-4739-9bef-b0ec4d4bb164" containerName="payload-processing"
Jul 28 15:28:23.537808 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537746    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e0bfb43-4a16-4739-9bef-b0ec4d4bb164" containerName="payload-processing"
Jul 28 15:28:23.537808 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537754    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c894837-74a6-4b3b-822b-67582d218ec1" containerName="maas-api"
Jul 28 15:28:23.537808 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537760    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c894837-74a6-4b3b-822b-67582d218ec1" containerName="maas-api"
Jul 28 15:28:23.537808 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537768    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="000c0b01-1123-4774-9aeb-f64eb32e595a" containerName="maas-api"
Jul 28 15:28:23.537808 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537773    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="000c0b01-1123-4774-9aeb-f64eb32e595a" containerName="maas-api"
Jul 28 15:28:23.537808 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537783    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1ab044b-c06b-46cb-b13f-b10885d9b835" containerName="payload-processing"
Jul 28 15:28:23.537808 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537789    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1ab044b-c06b-46cb-b13f-b10885d9b835" containerName="payload-processing"
Jul 28 15:28:23.537808 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537796    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33658123-a2fc-4f67-bb97-03711502555c" containerName="maas-api"
Jul 28 15:28:23.537808 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537801    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="33658123-a2fc-4f67-bb97-03711502555c" containerName="maas-api"
Jul 28 15:28:23.537808 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537807    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb7245e4-ecbf-465f-8203-54fb9db71ea0" containerName="payload-pre-processing"
Jul 28 15:28:23.537808 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537812    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb7245e4-ecbf-465f-8203-54fb9db71ea0" containerName="payload-pre-processing"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537822    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="733d7759-e53e-4aec-8878-333d5eec42a8" containerName="maas-api"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537828    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="733d7759-e53e-4aec-8878-333d5eec42a8" containerName="maas-api"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537833    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="279bbc3f-03c7-4f6f-a30b-daf6940e7c92" containerName="payload-pre-processing"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537839    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="279bbc3f-03c7-4f6f-a30b-daf6940e7c92" containerName="payload-pre-processing"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537850    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="344c75f5-e576-4598-a2c9-1ecfe0a7f7c6" containerName="payload-pre-processing"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537855    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="344c75f5-e576-4598-a2c9-1ecfe0a7f7c6" containerName="payload-pre-processing"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537862    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98a66db1-31b6-47aa-b879-1049d20e9877" containerName="payload-processing"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537867    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="98a66db1-31b6-47aa-b879-1049d20e9877" containerName="payload-processing"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537877    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8e46d34-99a8-4ef4-a26d-36783061146b" containerName="maas-api"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537881    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8e46d34-99a8-4ef4-a26d-36783061146b" containerName="maas-api"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537930    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1ab044b-c06b-46cb-b13f-b10885d9b835" containerName="payload-processing"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537938    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8e46d34-99a8-4ef4-a26d-36783061146b" containerName="maas-api"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537944    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="733d7759-e53e-4aec-8878-333d5eec42a8" containerName="maas-api"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537951    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e0bfb43-4a16-4739-9bef-b0ec4d4bb164" containerName="payload-processing"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537956    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c80aaa6e-877c-4437-9b94-72105d055102" containerName="istio-proxy"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537961    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="279bbc3f-03c7-4f6f-a30b-daf6940e7c92" containerName="payload-pre-processing"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537965    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="000c0b01-1123-4774-9aeb-f64eb32e595a" containerName="maas-api"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537985    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="344c75f5-e576-4598-a2c9-1ecfe0a7f7c6" containerName="payload-pre-processing"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537990    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb7245e4-ecbf-465f-8203-54fb9db71ea0" containerName="payload-pre-processing"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.537995    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="98a66db1-31b6-47aa-b879-1049d20e9877" containerName="payload-processing"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.538001    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c80aaa6e-877c-4437-9b94-72105d055102" containerName="istio-proxy"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.538006    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="33658123-a2fc-4f67-bb97-03711502555c" containerName="maas-api"
Jul 28 15:28:23.538268 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.538011    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c894837-74a6-4b3b-822b-67582d218ec1" containerName="maas-api"
Jul 28 15:28:23.541122 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.541079    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"
Jul 28 15:28:23.546896 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.546863    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hzqjc\"/\"kube-root-ca.crt\""
Jul 28 15:28:23.547075 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.547052    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hzqjc\"/\"openshift-service-ca.crt\""
Jul 28 15:28:23.550176 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.550155    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hzqjc\"/\"default-dockercfg-vtfl4\""
Jul 28 15:28:23.572110 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.569181    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"]
Jul 28 15:28:23.607635 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.607595    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8-sys\") pod \"perf-node-gather-daemonset-hlflb\" (UID: \"c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"
Jul 28 15:28:23.607635 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.607630    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8-proc\") pod \"perf-node-gather-daemonset-hlflb\" (UID: \"c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"
Jul 28 15:28:23.607889 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.607659    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnmqt\" (UniqueName: \"kubernetes.io/projected/c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8-kube-api-access-hnmqt\") pod \"perf-node-gather-daemonset-hlflb\" (UID: \"c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"
Jul 28 15:28:23.607889 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.607740    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8-lib-modules\") pod \"perf-node-gather-daemonset-hlflb\" (UID: \"c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"
Jul 28 15:28:23.607889 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.607850    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8-podres\") pod \"perf-node-gather-daemonset-hlflb\" (UID: \"c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"
Jul 28 15:28:23.708905 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.708860    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8-sys\") pod \"perf-node-gather-daemonset-hlflb\" (UID: \"c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"
Jul 28 15:28:23.708905 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.708900    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8-proc\") pod \"perf-node-gather-daemonset-hlflb\" (UID: \"c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"
Jul 28 15:28:23.709147 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.708978    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8-proc\") pod \"perf-node-gather-daemonset-hlflb\" (UID: \"c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"
Jul 28 15:28:23.709147 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.708981    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8-sys\") pod \"perf-node-gather-daemonset-hlflb\" (UID: \"c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"
Jul 28 15:28:23.709147 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.709018    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnmqt\" (UniqueName: \"kubernetes.io/projected/c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8-kube-api-access-hnmqt\") pod \"perf-node-gather-daemonset-hlflb\" (UID: \"c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"
Jul 28 15:28:23.709147 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.709051    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8-lib-modules\") pod \"perf-node-gather-daemonset-hlflb\" (UID: \"c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"
Jul 28 15:28:23.709283 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.709184    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8-podres\") pod \"perf-node-gather-daemonset-hlflb\" (UID: \"c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"
Jul 28 15:28:23.709283 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.709254    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8-lib-modules\") pod \"perf-node-gather-daemonset-hlflb\" (UID: \"c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"
Jul 28 15:28:23.709348 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.709298    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8-podres\") pod \"perf-node-gather-daemonset-hlflb\" (UID: \"c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"
Jul 28 15:28:23.719645 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.719605    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnmqt\" (UniqueName: \"kubernetes.io/projected/c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8-kube-api-access-hnmqt\") pod \"perf-node-gather-daemonset-hlflb\" (UID: \"c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"
Jul 28 15:28:23.850894 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.850866    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"
Jul 28 15:28:23.997123 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:23.997070    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"]
Jul 28 15:28:24.000809 ip-10-0-130-66 kubenswrapper[2584]: W0728 15:28:24.000779    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc627ff4c_3ff6_4c2f_9b1c_bb3445a2c6b8.slice/crio-b16275f23a6f0862f00950560f7a6d6b364d2dc4d377d54e238c01a08266214e WatchSource:0}: Error finding container b16275f23a6f0862f00950560f7a6d6b364d2dc4d377d54e238c01a08266214e: Status 404 returned error can't find the container with id b16275f23a6f0862f00950560f7a6d6b364d2dc4d377d54e238c01a08266214e
Jul 28 15:28:24.540073 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:24.539992    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pc4z2_1f42b3d4-b9fa-4e4d-b85d-99d733a93f11/node-exporter/0.log"
Jul 28 15:28:24.577200 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:24.577176    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pc4z2_1f42b3d4-b9fa-4e4d-b85d-99d733a93f11/kube-rbac-proxy/0.log"
Jul 28 15:28:24.616266 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:24.616236    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pc4z2_1f42b3d4-b9fa-4e4d-b85d-99d733a93f11/init-textfile/0.log"
Jul 28 15:28:24.873031 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:24.873002    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb" event={"ID":"c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8","Type":"ContainerStarted","Data":"f28eea0b2e612903137ef733ac36bafb85e33a59df5ea0334d29af723d386c93"}
Jul 28 15:28:24.873221 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:24.873051    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb" event={"ID":"c627ff4c-3ff6-4c2f-9b1c-bb3445a2c6b8","Type":"ContainerStarted","Data":"b16275f23a6f0862f00950560f7a6d6b364d2dc4d377d54e238c01a08266214e"}
Jul 28 15:28:24.873221 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:24.873118    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"
Jul 28 15:28:24.894063 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:24.894013    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb" podStartSLOduration=1.8939932499999999 podStartE2EDuration="1.89399325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 15:28:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:28:24.89260049 +0000 UTC m=+4174.694142706" watchObservedRunningTime="2026-07-28 15:28:24.89399325 +0000 UTC m=+4174.695535471"
Jul 28 15:28:25.100588 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:25.100562    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-8g9dz_1616daad-693b-483b-a760-015508392f41/prometheus-operator-admission-webhook/0.log"
Jul 28 15:28:28.324375 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:28.324342    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-j7vw6_ebaf4287-a41e-478b-b913-ec6e7a66cbb4/volume-data-source-validator/0.log"
Jul 28 15:28:29.469263 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:29.469230    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hkt4c_8dc38f88-df97-4d9f-8718-306934c61e2b/dns/0.log"
Jul 28 15:28:29.495391 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:29.495364    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hkt4c_8dc38f88-df97-4d9f-8718-306934c61e2b/kube-rbac-proxy/0.log"
Jul 28 15:28:29.620314 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:29.620277    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k66dk_d3c1ce2b-ee28-4657-81d3-a2f28736fab4/dns-node-resolver/0.log"
Jul 28 15:28:30.236377 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:30.236343    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-d9f895f7-79bkf_9b8f57f2-979c-4920-a974-e4de894b8b62/registry/0.log"
Jul 28 15:28:30.295398 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:30.295365    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jldlr_1c2b854d-bbb4-4b63-b880-f71b7df99fdd/node-ca/0.log"
Jul 28 15:28:30.887876 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:30.887848    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-hlflb"
Jul 28 15:28:31.406231 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:31.406199    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr_a82f1155-e65d-46c1-aa7a-739022d05fc5/istio-proxy/0.log"
Jul 28 15:28:31.568761 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:31.568728    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-4ndsh_e8929d68-d91e-4163-bbfc-f848bba1f652/discovery/0.log"
Jul 28 15:28:31.626108 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:31.626078    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c4fff746-6ksdx_4e108199-9380-42ba-8d02-8361be3b37ad/kube-auth-proxy/0.log"
Jul 28 15:28:31.659449 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:31.659377    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c747fcbf8-4nzkp_0341cb69-6aaa-47f2-8009-28908c0830c7/istio-proxy/0.log"
Jul 28 15:28:31.685193 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:31.685167    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c747fcbf8-4nzkp_0341cb69-6aaa-47f2-8009-28908c0830c7/istio-proxy/1.log"
Jul 28 15:28:31.719146 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:31.719118    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-82vf5_9566b9fe-f896-4d0a-a68a-823856c7aa06/payload-pre-processing/0.log"
Jul 28 15:28:31.750810 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:31.750783    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-2hwhh_5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2/payload-processing/0.log"
Jul 28 15:28:32.880228 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:32.880191    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m2qz5_e78ad162-6c3b-41de-ad16-9f39ce98f98e/serve-healthcheck-canary/0.log"
Jul 28 15:28:33.801353 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:33.801321    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cbx4k_d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d/kube-rbac-proxy/0.log"
Jul 28 15:28:33.832698 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:33.832668    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cbx4k_d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d/exporter/0.log"
Jul 28 15:28:33.862084 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:33.862055    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cbx4k_d1fe4fd0-9976-4a2a-b23f-bd54da1bc59d/extractor/0.log"
Jul 28 15:28:34.294388 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:34.294360    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr_a82f1155-e65d-46c1-aa7a-739022d05fc5/istio-proxy/0.log"
Jul 28 15:28:34.435179 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:34.435143    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-4ndsh_e8929d68-d91e-4163-bbfc-f848bba1f652/discovery/0.log"
Jul 28 15:28:34.473817 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:34.473795    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c4fff746-6ksdx_4e108199-9380-42ba-8d02-8361be3b37ad/kube-auth-proxy/0.log"
Jul 28 15:28:34.493613 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:34.493591    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c747fcbf8-4nzkp_0341cb69-6aaa-47f2-8009-28908c0830c7/istio-proxy/0.log"
Jul 28 15:28:34.522758 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:34.522739    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c747fcbf8-4nzkp_0341cb69-6aaa-47f2-8009-28908c0830c7/istio-proxy/1.log"
Jul 28 15:28:34.564456 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:34.564394    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-82vf5_9566b9fe-f896-4d0a-a68a-823856c7aa06/payload-pre-processing/0.log"
Jul 28 15:28:34.646472 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:34.646444    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-2hwhh_5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2/payload-processing/0.log"
Jul 28 15:28:35.960002 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:35.959974    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6cbcdcfcccl7qbr_a82f1155-e65d-46c1-aa7a-739022d05fc5/istio-proxy/0.log"
Jul 28 15:28:36.119141 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:36.119101    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-4ndsh_e8929d68-d91e-4163-bbfc-f848bba1f652/discovery/0.log"
Jul 28 15:28:36.159220 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:36.159187    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c4fff746-6ksdx_4e108199-9380-42ba-8d02-8361be3b37ad/kube-auth-proxy/0.log"
Jul 28 15:28:36.182917 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:36.182887    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c747fcbf8-4nzkp_0341cb69-6aaa-47f2-8009-28908c0830c7/istio-proxy/0.log"
Jul 28 15:28:36.212614 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:36.212520    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c747fcbf8-4nzkp_0341cb69-6aaa-47f2-8009-28908c0830c7/istio-proxy/1.log"
Jul 28 15:28:36.247826 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:36.247780    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-82vf5_9566b9fe-f896-4d0a-a68a-823856c7aa06/payload-pre-processing/0.log"
Jul 28 15:28:36.270188 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:36.270155    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-2hwhh_5e4aacc1-0e99-4a7e-81fb-04bc4ca5def2/payload-processing/0.log"
Jul 28 15:28:37.064009 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:37.063969    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-65db4695c7-z9h72_ddf405d5-caef-4e8f-8882-3d5a12a9d41e/manager/0.log"
Jul 28 15:28:37.101169 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:37.101143    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dbm5c_14c17f23-5dac-413e-9df4-bf19cfa8c4ef/server/0.log"
Jul 28 15:28:37.207256 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:37.207227    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-fwf8p_e360876b-132e-4a38-bbeb-d6c87c9b8548/manager/0.log"
Jul 28 15:28:37.291272 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:37.291235    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65f7956c87-29zbg_80778efc-8a53-41a7-b6b3-dd27455c0004/manager/0.log"
Jul 28 15:28:37.313919 ip-10-0-130-66 kubenswrapper[2584]: I0728 15:28:37.313890    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65f7956c87-29zbg_80778efc-8a53-41a7-b6b3-dd27455c0004/copy-manifests/0.log"